{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 7. Naive Bayes\n",
    "### Agenda\n",
    "0. Generative vs Discriminative Models\n",
    "1. Introduction Bayes' Theorem\n",
    "2. Maximum Likelihood Estimate\n",
    "3. Naive Bayes Classifier\n",
    "4. Gaussian Naive Bayes\n",
    "5. Multinomial Naive Bayes\n",
    "5. Burnolis' Naive Bayes\n",
    "6. Naive Bayes for out-of-core "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Discriminative vs Generative\n",
    "\n",
    "![](img/7-generative-vs-discriminative-01.jpg)\n",
    "\n",
    "## 7.1 Introduction to Naive Bayes\n",
    "* The Naive Bayes Classifier technique is based on the Bayesian theorem and is particularly suited when then high dimensional data. \n",
    "* It's simple & out-performs many sophisticated methods\n",
    "* Rather than attempting to calculate the values of each attribute value P(d1, d2, d3|h), they are assumed to be conditionally independent given the target value and calculated as P(d1|h) * P(d2|h) and so on.\n",
    "* The above assumption is very strong & not true for in real situations, still naive bayes works quite well"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 7.1.1 Class Probabilities\n",
    "* For Bi-class classification, P(Class 1) = Count(Class 1) / Count( Class 1 + Class 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 7.1.2 Conditional Probabilities\n",
    "* Frequency of each attribute value for each class\n",
    "* Consider a dataset with attribute - weather ( values - sunny & rainy ). Target - Sports ( values - chess & tennis )\n",
    "* P(weather=sunny|target=tennis) = Count ( weather=sunny & target=tennis ) / Count ( target=tennis )\n",
    "![](img/venn-diagram-01.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7.2 Naive Bayes' Classifier\n",
    "* Formula : Prediction = Max(P(feature|h).P(h))\n",
    "* Let's predict for a new data (weather=sunny)\n",
    "* Possibility of tennis = P(weather=sunny|target=tennis) . P(target=tennis)\n",
    "* Possibility of chess = P(weather=sunny|target=chess) . P(target=chess)\n",
    "* We choose the possibility with higher values\n",
    "* Normalize the value to bring it to scale of 0 to 1\n",
    "\n",
    "#### More features\n",
    "* In case, we add more feature like skill (values - low,moderate,high)\n",
    "* Our probability becomes, P(weather=sunny|target=tennis).P(skill=moderate|target=tennis).P(target=tennis)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7.2.1 Gaussian Naive Bayes\n",
    "* The above fundamental example is for categorical data\n",
    "* We can use Naive Bayes for continues data as well\n",
    "* Assumption is data should be of Gaussian Distribution\n",
    "* Let's understand a bit about Gaussian PDF\n",
    "\n",
    "$$P(x|\\mu,\\sigma) = g_{\\mu,\\sigma}(x) = \\cfrac{1}{\\sqrt{2\\pi\\sigma^2}}e^{-\\cfrac{(x-\\mu)^2}{2\\sigma^2}}$$\n",
    "\n",
    "![](img/7-gaussian-01.jpg)\n",
    "* Possibility of tennis = P(pdf(precipitation)|class=tennis) . P(pdf(windy)|class=tennis) . P(class=tennis\n",
    "* Prior probability can be configured. By default, each class is assigned equal probability\n",
    "#### Temperature Distribution\n",
    "The distribution of temperature in city can be a good example of Gaussian Distribution. \n",
    "![](img/7-temperature-distribution-01.jpg)  \n",
    "Taking the above Temperature Distribution Example, let us try to understand the two fundamental terms in Bayesian Inference.  \n",
    "1. **Likelihood** ($P(X|\\mu,\\sigma)$) : How likely is the data generated from this guassian.This is given by $ \\Pi_{i=0}^{N} g(x_i)$ which is joing probability.\n",
    "2. **Posterior**  ($P(\\mu,\\sigma|X)$)  : How likely is the gaussian with $\\mu$ mean representative of data's distribution. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.datasets import load_iris\n",
    "import seaborn as sns; sns.set(color_codes=True)\n",
    "from mpl_toolkits.mplot3d import Axes3D\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "iris = load_iris()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame(iris.data, columns=iris.feature_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.naive_bayes import GaussianNB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal length (cm)</th>\n",
       "      <th>sepal width (cm)</th>\n",
       "      <th>petal length (cm)</th>\n",
       "      <th>petal width (cm)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.6</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5.0</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   sepal length (cm)  sepal width (cm)  petal length (cm)  petal width (cm)\n",
       "0                5.1               3.5                1.4               0.2\n",
       "1                4.9               3.0                1.4               0.2\n",
       "2                4.7               3.2                1.3               0.2\n",
       "3                4.6               3.1                1.5               0.2\n",
       "4                5.0               3.6                1.4               0.2"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "gnb = GaussianNB()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GaussianNB(priors=None, var_smoothing=1e-09)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gnb.fit(df,iris.target)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.96"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gnb.score(df,iris.target)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Multinomial Naive Bayes\n",
    "* Suited for classification of data with discrete features ( count data )\n",
    "* Very useful in text processing\n",
    "* Each text will be converted to vector of word count\n",
    "* Cannot deal with negative numbers\n",
    "![](img/7-movie-review-01.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Amazon Food Review Sentiment Prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "food_review = pd.read_csv('C:\\\\Users\\\\jaley\\\\git\\\\AI102-MachineLearning\\\\data\\\\reviews.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 568454 entries, 0 to 568453\n",
      "Data columns (total 10 columns):\n",
      "Id                        568454 non-null int64\n",
      "ProductId                 568454 non-null object\n",
      "UserId                    568454 non-null object\n",
      "ProfileName               568438 non-null object\n",
      "HelpfulnessNumerator      568454 non-null int64\n",
      "HelpfulnessDenominator    568454 non-null int64\n",
      "Score                     568454 non-null int64\n",
      "Time                      568454 non-null int64\n",
      "Summary                   568427 non-null object\n",
      "Text                      568454 non-null object\n",
      "dtypes: int64(5), object(5)\n",
      "memory usage: 43.4+ MB\n"
     ]
    }
   ],
   "source": [
    "food_review.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Text</th>\n",
       "      <th>Score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>I have bought several of the Vitality canned d...</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Product arrived labeled as Jumbo Salted Peanut...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>This is a confection that has been around a fe...</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>If you are looking for the secret ingredient i...</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Great taffy at a great price.  There was a wid...</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                Text  Score\n",
       "0  I have bought several of the Vitality canned d...      5\n",
       "1  Product arrived labeled as Jumbo Salted Peanut...      1\n",
       "2  This is a confection that has been around a fe...      4\n",
       "3  If you are looking for the secret ingredient i...      2\n",
       "4  Great taffy at a great price.  There was a wid...      5"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "columns = ['Text','Score']\n",
    "food_review[columns].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "review_data = food_review[['Text','Score']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "review_data = review_data[review_data.Score != 3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "review_data['Sentiment'] = review_data.Score.map(lambda s:0 if s < 3 else 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "review_data.drop('Score',axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Text</th>\n",
       "      <th>Sentiment</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>I have bought several of the Vitality canned d...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Product arrived labeled as Jumbo Salted Peanut...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>This is a confection that has been around a fe...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>If you are looking for the secret ingredient i...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Great taffy at a great price.  There was a wid...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                Text  Sentiment\n",
       "0  I have bought several of the Vitality canned d...          1\n",
       "1  Product arrived labeled as Jumbo Salted Peanut...          0\n",
       "2  This is a confection that has been around a fe...          1\n",
       "3  If you are looking for the secret ingredient i...          0\n",
       "4  Great taffy at a great price.  There was a wid...          1"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "review_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    443777\n",
       "0     82037\n",
       "Name: Sentiment, dtype: int64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "review_data.Sentiment.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "review_data = review_data.sample(10000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Remove punchuations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from nltk.tokenize import RegexpTokenizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "tokenizer = RegexpTokenizer(r'[A-Za-z]+')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "review_data['Text'] = review_data.Text.map(lambda x:tokenizer.tokenize(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "230837    [Haven, t, bought, this, online, just, at, the...\n",
       "272358    [If, you, have, never, tried, it, you, must, I...\n",
       "515179    [I, found, this, brand, in, my, local, grocery...\n",
       "529006    [I, bought, these, for, my, friend, in, the, U...\n",
       "154693    [My, son, has, been, using, Prilosec, for, at,...\n",
       "367246    [I, picked, up, a, box, of, Special, K, red, b...\n",
       "531731    [I, love, this, coffee, It, is, wonderful, smo...\n",
       "439844    [Excellent, item, loved, by, both, my, fussy, ...\n",
       "532916    [I, love, PB, It, tastes, just, like, peanut, ...\n",
       "474071    [The, white, tea, blended, w, the, green, take...\n",
       "93923     [My, english, bulldogs, love, these, cookies, ...\n",
       "311536    [My, dogs, like, these, the, best, and, I, m, ...\n",
       "557166    [I, bought, the, Kettle, Corn, variety, Pop, U...\n",
       "225998    [Very, Very, overpriced, Starbucks, K, Cup, fo...\n",
       "205138    [After, trying, a, few, different, treats, for...\n",
       "100424    [BLEND, this, for, a, Great, Smooth, cup, of, ...\n",
       "53279     [From, a, former, Dr, Pepper, drinker, I, have...\n",
       "501236    [After, realizing, I, was, buying, these, just...\n",
       "158313    [My, kids, have, been, eating, these, as, a, h...\n",
       "378319    [The, Coffee, People, Jet, Fuel, K, Cups, for,...\n",
       "45303     [These, premo, treats, are, getting, harder, t...\n",
       "457311    [Great, product, I, use, them, to, add, to, re...\n",
       "210386    [Our, Maine, Coons, and, Ragdolls, love, this,...\n",
       "499803    [I, despise, low, fat, non, fat, dressings, an...\n",
       "355328    [Did, not, know, that, it, had, to, be, mixed,...\n",
       "548163    [A, small, amount, to, the, scalp, and, tips, ...\n",
       "126094    [he, ll, actually, ask, for, it, we, can, only...\n",
       "80224     [Canidae, has, changed, the, formula, of, thei...\n",
       "151055    [My, little, one, loves, these, and, you, cann...\n",
       "14159     [Boy, did, I, ever, test, this, out, br, br, E...\n",
       "                                ...                        \n",
       "73647     [This, is, the, best, granola, I, ve, ever, fo...\n",
       "41846     [I, HAVE, FOUND, THIS, SAUCE, THE, FRESHEST, A...\n",
       "310982    [I, ve, been, buying, Bariani, Olive, Oil, for...\n",
       "394899    [i, have, this, strong, flavorful, mahogany, c...\n",
       "472562    [Packaged, in, individual, pouches, similar, t...\n",
       "109357    [I, love, this, brand, of, whole, grains, I, h...\n",
       "566162    [This, product, arrived, in, good, time, and, ...\n",
       "386072    [This, was, my, first, mate, de, coca, product...\n",
       "18712     [My, lab, loves, anything, Zukes, He, will, do...\n",
       "395292    [These, are, the, BEST, dried, cranberries, ou...\n",
       "46995     [I, bought, this, because, it, s, lower, in, c...\n",
       "498850    [Turned, on, to, these, by, a, friend, who, re...\n",
       "272921    [I, decided, to, try, a, couple, of, brands, o...\n",
       "284322    [Baby, Gourmet, Juicy, Pear, Garden, Greens, i...\n",
       "432516    [I, think, they, are, a, great, product, in, t...\n",
       "242221    [Bought, this, for, my, wife, she, needed, it,...\n",
       "68788     [This, is, a, really, great, product, First, t...\n",
       "66703     [Best, tea, available, in, the, market, have, ...\n",
       "15128     [It, s, a, little, runny, but, could, not, tas...\n",
       "327252    [this, stuff, from, which, culantro, cilantro,...\n",
       "238416    [This, product, is, Quick, Easy, Smooth, Inexp...\n",
       "29298     [I, love, this, Popcorn, We, use, it, at, the,...\n",
       "549026    [I, like, this, tea, I, can, drink, it, with, ...\n",
       "396315    [Hershey, s, Sugar, Free, Dark, Chocolate, can...\n",
       "106615    [I, always, drank, community, coffee, when, I,...\n",
       "51658     [Very, hot, and, perfect, for, toothpicks, Gre...\n",
       "522419    [I, was, thoroughly, pleased, with, the, flavo...\n",
       "17912     [We, discovered, a, cafe, that, had, excellent...\n",
       "21473     [My, small, dog, loves, his, greenies, I, give...\n",
       "243290    [I, love, Click, I, don, t, know, what, I, d, ...\n",
       "Name: Text, Length: 10000, dtype: object"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "review_data.Text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Stemming\n",
    "A **root** is a form which is not further analysable In the form ‘untouchables’ the root is ‘touch’, to which first the suffix ‘-able’, then the prefix ‘un-‘ and finally the suffix ‘-s’ have been added. In a compound word like ‘wheelchair’ there are two roots, ‘wheel’ and ‘chair’.\n",
    "![](img/7-word-dissection.png)\n",
    "**Derivational Morphology** Derivational morphology results in the creation of a new word with a new meaning ![](img/7-derivational.png)\n",
    " \n",
    "**Inflictional Morphology** In contrast, inflectional morphology involves an obligatory grammatical specification \n",
    "![](img/7-inflictional.png)\n",
    "\n",
    "A **stem** is of concern only when dealing with **inflectional morphology**.(which doesn't change the core meaning)\n",
    "In the form *‘untouchables’* the stem is *‘untouchable’*, although in the form *‘touched’* the stem is *‘touch’*; in the form *‘wheelchairs’* the stem is *‘wheelchair’*, even though the stem contains two roots('wheel' and 'chair').\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "from nltk.stem.snowball import SnowballStemmer\n",
    "stemmer = SnowballStemmer(\"english\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "review_data['Text'] = review_data.Text.map(lambda l: [stemmer.stem(word) for word in l])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "review_data.Text = review_data.Text.str.join(sep=' ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Preprocessing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "cv = CountVectorizer(stop_words='english')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "review_data_tf = cv.fit_transform(review_data.Text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(10000, 13154)\n"
     ]
    }
   ],
   "source": [
    "print (review_data_tf.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Splitting data into train_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split;\n",
    "trainX,testX,trainY,testY = train_test_split(review_data_tf,review_data.Sentiment)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Create Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5    8471\n",
       "0    1529\n",
       "Name: Sentiment, dtype: int64"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "review_data.Sentiment.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Class Imbalanced\n",
    "* Two ways to handle this - dealing with data, dealing with algo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.naive_bayes import MultinomialNB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "mnb = MultinomialNB(class_prior=[.25,.75])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MultinomialNB(alpha=1.0, class_prior=[0.25, 0.75], fit_prior=True)"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mnb.fit(trainX,trainY)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.25, 0.75]"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mnb.class_prior"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = mnb.predict(testX)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8924"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mnb.score(testX,testY)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import confusion_matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 203,  180],\n",
       "       [  89, 2028]], dtype=int64)"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "confusion_matrix(y_true=testY, y_pred=y_pred)\n",
    "# Rows corrospond to the actual classes\n",
    "# Collumns corrospond to the predicted classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bernoulli's Naive Bayes\n",
    "* Like MultinomialNB, this classifier is suitable for discrete data.\n",
    "* The difference is that while MultinomialNB works with occurrence counts\n",
    "* BernoulliNB is designed for binary/boolean features.\n",
    "* If data is not binary, internally Binarization preprocessing will happen\n",
    "* Can deal with negative numbers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import make_classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "X, Y = make_classification(n_samples=500, n_features=2, n_informative=2, n_redundant=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x24b12ce5cc0>"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEBCAYAAACdctWRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzsnXd4FNX6xz/nzNY0kkDo1cKCFNFrQ0DF9rP3gnrt5SoWrr3ce+3dK/be67WjAnbsCIIISF06CERaerbPOb8/ZrPJklADaZzP8/CQ3Zkz885s8p2z73mL0FpjMBgMhpaJbGwDDAaDwbD9MCJvMBgMLRgj8gaDwdCCMSJvMBgMLRgj8gaDwdCCMSJvMBgMLRgj8gaDwdCCMSJvMBgMLRgj8gaDwdCCMSJvMBgMLRgj8gaDwdCCcTXCOb3A3kAhYDfC+Q0Gg6E5YgEdgMlAdHMHNYbI7w381AjnNRgMhpbAEODnzd25MUS+EKC4uBKlNlwBs3XrLNatq2gwo7YFxuaGoTnaDM3TbmNzw7A5NkspyMvLhKSGbi6NIfI2gFJ6oyJftU9zw9jcMDRHm6F52m1sbhi2wOYtcnObhVeDwWBowRiRNxgMhhaMEXmDwWBowRiRNxgMhhaMEfkmjtDFoCONbYbBYGimGJFvqmhFln0jeepI8tUhuPSkxrbIYDA0Q4zIN1FczMDDeARxBBGy1L2NbZLBYGiGGJFvouj1Uhg07kayxGAwNGfqlQwVCATuBE4BNPBSMBgcuU2sMmCzGxGOwceHaDKplLc2tkkGg6EZstUiHwgEDgQOBvoDbmB2IBAYGwwGg9vKuB0aIQhZNxHS1wEWCNHYFhkMhmbIVrtrgsHgD8DQYDCYANriPDAqt5VhhiTCZQTeYDBsNfXyyQeDwXggELgDmA2MA1ZsE6sMBoPBsE0QWte/kE8gEMgARgPvBoPB5zexe3dgcb1PajAYDDsmPYAlm7tzfXzyvQBfMBicFgwGQ4FA4CMc//xmsW5dxUarrhUUZLNmTfnWmtcoGJsbhuZoMzRPu5uDzWuWraWyLETX3TojpWwWNq/P5tgspaB166wtPnZ9omt2Au4IBAKDcaJrjgdersfxDAaDYYv4+tXvePO2d5BS0nPvXbjxnasb26QmR30WXj8DxgJTgSnAL8Fg8J1tZZjBYDBsivfvH0U8EicaijJv8gKWzV7e2CY1OeoVJx8MBm8Hbt8mlhgMBsMWkpWbQUWx01FJKUVmq4xGtqjpYTJeDQZDs2XES8Np36MtWXlZnHPXGRR0adPYJjU5GqP9n8FgMGwTuvXpysiJ9ze2GU0aM5M3GAyGFowR+eaMDoFWjW2FwWBowhh3TTND6CLcejw+/Q4u5qPJpVS+gBLdGts0g8HQBDEi34wQuphcdRqCSiCGU9GmiEz1KOXWI41rnMFgaJIYd00TwtLzaWWfRJ59KF71ca3tbj0JiCJSAl9F/UtTGAyGlokR+SZEtroeF0uRFJOpH0TqwrTttuiCwPHBawQa0ORSKf/ZCNYaDIbmgHHXNCEEJTVeSQTlQIfUO7bYjQpxM379PxLsREhcgRYFIMyz2mAw1I0R+SZESFxKpn4cjSRBf2x2qbVPTB5DjGNqD9Y2FotQtEaL/Aaw1mAwNAeMyDchonIYcT0IQQU2gc2foes4OepCXCwEEkTEKYTEP0GYvrAGw46O+Z7fxFCiC279M3nqSLLtfyB00SbHuJmMxSIEEQQJfPpdstW1DWCtwWBo6hiRb2K49GQy9GtI1uJmKpnqvk2O0WSnFmQBBBo3v8A2aAhjMBiaN0bkmxhSr00FRApsJKsAsPRs/PbTePQ3tcQ7IfoRZlgy2gY0EkVX0xvWYDAYn3xTIy4Go3UuConAJiwvcuLn1cVABJSPSrGOqDg9bVzYuoqYPha/egaNl7C8snEuwGAwNCmMyDcxtMimRL6Pi1koOqJER3zqXUAlE6AiePQ44nooiry0xVVb9KDCerCRLDcYDE0R465pigg/CbEXSnQEIC76pjZpwM3v5KrjyVNH10qYMhgMhpoYkW8G2KIPcfqjITmb18nSBsX41UuNa5zBYGjSGJFvJii6ANZ670o0nsYwx2AwNBOMyDcTQnI4Nj2SNWuy0QhsuhGWFze2aQaDoQljFl6bAVIX4levEWc/ysSzaJkH2gZhpe3j1hOxxS4kRL9GtNZgMDQl6iXygUDgNuC05MuxwWDwhvqbZEhDx2mlzkNQBEjcejKlvL2ewK+glToDsBEaKsStxOT/NZrJBoOh6bDV7ppAIHAocDiwBzAA+FsgEDhxWxlmcJCsQ1COQCFIYDGvVss/t/4FQQJJBEEEn/6okaw1GAxNjfr45AuBa4PBYCwYDMaBOUDXbWOWoQpFG6eyJC40HhL0r1W4zBY9qI678ZKgVyNYajAYmiJb7a4JBoOzqn4OBAK74rhtBm0Loww1EC5K5ev49AdovETEqbV2SYi9qBQ34NWfkGA3QvLyRjDUYDA0RYSuZxGrQCDQBxgL3BYMBl/bjCHdgcX1OqkBAK1tCH+Etlcj/McjXJ0b2ySDwbD96QEs2dyd67vwOgj4EPhnMBh8Z0vGrltXgVIbfsAUFGSzZk15fcxrcBra5kz7Hrx8BsRRla9QLh5EsiaZLdt2s45h7nPD0RztNjY3DJtjs5SC1q2ztvjYWy3ygUCgC/AxcHowGPx2a49j2Hrc/IQgknwVI0dfCbjQ2qJUvoMSHTY23GAw7ADUZ+H1OsAHjAwEAtOS/y7dRnYZNoMEu6PxJEsTxxDEEYQRxHHrnxrZOoPB0BSoz8LrCGDENrTFsIVUyDvw61eQeiWS1biZjiAOCJTo1tjmGQyGmugoFsuw6Qgis8FOa8oaNGeEj7D4BxaFuJgFKGxaE+UQPOpTPGpMY1toMBgAoYvIVSeQoy4gTx2NpRc12LmNyDdzLJbiYg6ScLKTVBlexuHjC7L0PeTYZ+O3nwRlN7apBsMOi1ePRlKEJISgAp96tcHObUS+mSJ0CS49Da1dkPTKO8XLPKnFWEEMF7Px8wo5+uxGtNZg2LHRZFHtHbfQ5DTYuU2BsmaIpReSo86vekWluA6/fh1FNlFxMpn6ISCCoCoPFlwEG8dYg6EZInQRYKFFq21yvKg4DreegIcJJOhFWP5jmxx3czAi3wzxqXcQhBBoNBKpV1FifZzabutOZKurEYSBqubefiw9B5ueacXNDIbGRtmKt+98j6lfT6fP4N6cc8+ZuNyNJ01++wn8vAVApfgnUTms/gcVbiqs/9b/OFuBcdc0Q5RoA1T1dnWjRGukXomlZ4FOkKAPcfZIum9A40OQIEddTLYajlCrcelpoEONeBUGg8MP//uZb177nsKFq/jx3V/44vmvAZB6FVn2zWTb/8TS8xvEFqFL8PNWMhw5TqZ+pFZBwOaGmck3Q8LiXCw9DzfTiLEfWnvI1acAkgS7onHjZgqCKm+9RBBBEMfFVPL0UaBdaHLQ9mjA25iXY9jBWbV0NbFIDIBEPErZmkW41ESy9TUIogDkqCkUy69B+LazNet/y3VR7fRsnhiRb44IHxXWw6mXufaRqT8GiwWAnfq1dIRepfrDitS2OFAMkc8AUyHasGni0Tijn/yMVUvWcNj5B7PLnjttk+MOPnkgX700joKOMR54dzZ5bWcj9CvJ76FVhJB6LUps3/pMWmRTKa4nU/8XsCgXd4MwIm9oZDR5aNYg0AhCyfdIzeSjHISbuVi1ahoptA43rLGGZoXWCo/6DMlqnru+nAmfzCQWiTNpzBQe/PEuCrq0qfc5OvfqxEM/34M3ejf57WyEUNRV1UqxbRZBN0VUnkxUn+S8aOYCD8Yn32zwqk/Itq/Cp96s5SMsl/eiyK8xW0/Hx5c4X0PFetsEyPbpO2vltBY07DhoDRt42Ovy+8jS95KhnyY4YRKxSBwAaUn+nLN8M4+vnHNshNYd82nbtVNarwSd+ieJMRRk9uadb1sgRIsQeDAi3yxw6+/J1A/iYTwZ+hm8+oP19vAiKavTc+iIvsZiEYp2yeYjVC/Iegem9vWor8lXg8hX++NV722/CzI0GaReRa46lnx1ADn2uaDW4VWjcevvHHGOfJmsh2Qz8P8q8fpdCOn8pu00oMcmj+9Xz5Kv9iNPDcWlp25037C8AJuuaAQJehHiMkLiEsrEk1TIBwFw6Rlk2I/hUZ9t8sFhcDDummaAS82FVIJTBJeeQTTVWpfktkTqla7xf7VvXqNojSIbF/OT7yl0xdNk2WsRFOFmGiJ5nEw9kpg+Ci22vLSpofngV88iWY1A4WIeeXqY4/LTCRLsAt5+6GgRghgX37aadn0uYvXyGINOHkhu2xruE20j9GokpSjRCS2ykXopfv16ch2onCz1b0qssRu0RYs8Sq33QSdA1JYmSy8gR12KIILWPkKsIyJMkt+mMCLfDIjLwfjVG2gSgIuYPCxtu6I7Gn/KHw+1XTYasOmGl69qbItC+G08VPvv00c079CxHQ2pC/GrFwBJSP4DLQrq3M+lJ+FWk0nIv+F8xtXTAqefsOOScTEXEpIwZ2CxkojrLA44o1+d581R5yJZBwi0zqBMvkz6NMPJwN4s6hB4x+6pKVsFETz6RyIYkd8URuSbAQnRl1L5Im79GwnRl4TYI30HIajkX2RxBxqJII5OE2k35dyOl3HUnPGnhtf43/kTsgiJC9Gi4VKvDfVE2+SoC5CsBQRuNYkS+Uktv7JbTyBbXQtEQb1NhbgRt56EpAibblgsh6TICwB7DmH5Wq2+wjXxqxeQFKW6DEMlPvUKlfJuohyCl68AQaW4easvT+rluPTvVP3+anzExH5bfbwdCSPyzQRb9MYWvTe4PWYdQYnui2QNSrfGr18CIMzfEUJhiy5kqpFpM/yaETg151sl4iOU7Aw67Hzd1uVE5Oko0WW7XJuh/gjKkRQjkg92SSEQI5UDoUO49Xiy9N2pcFuI4NazKJFjEVSgycalp5Kjh+MIvQth7USm/QCKDCLyArSoa/FzfRmRaHKdyYd1JyE9Ao1v68vr6jCt1LkISgELm9aExSVExUlbd7wdDCPyLQglOqPoDAIquQOp/6KV+jvoKGg368/i63LpaPwgnGzabHUdbn4H4njVZxTLMSAyGuJSdjiELiNT3YnFEiLiDKLy5C0ar2mVnIkvAyBBTxDe5LHLaaVOT7pTaq7duImLv4GQqYJZCbEnxXoMPvU2YJFhf4SXDwEXbjWVMutV0AksPQOBRUL0ISQvwa0mIfkTJyGvH2FZ3T9Ii9b1uDNgJR9YTtx8Akk5UXlKvY65I2FEvqWhNZJCNH686n0Epc4CK4I4/ZONRdJJn81HyFS3U249k9zX8aNqElgsd2rfGLY5meouPPyEIEGmHomte5IQtf3fdaJD5KjLsFgIWEQ5lkp5bWqzW/+EoCy1qA5OWGKUw4jJw2sfTrQhbF2F1MvIUP9LimvcKWmtV5GdOI3Vf8bIbWPjz+xPuXzeqZ2k46kJwrbEpiPgRxMBXMTZc5ufoyVjRL4loTVZ6hY8fAc4SVBOlKwCBAn2AcDFHASxlNjXXHAVKCRrAIizJ24m48z+PNgYd81moaPJjl2FRORp2KLPRnf3qk/w8G2Nz0MgdSHUIfJSr8CvXkHjIywvRIs8MtUDuJiZHJ/AyygSaheillNYS4m2CF11bACJogsxcQS59jEI4lSIm4iLA/HqMUi9nJg8Elu3BuFF6xhgEWcAMvQKV5/UgaVBH1Jq7vnfAjrtsxCbXbeLwAMgfJTKN/GqD9Aii4g4bdNjDCmMyLcgLD0PD1+n0sE9/ED14qvCwzeUiRfI0I/jZTTVdejdaNoiKEZgExZOGdRy+SA+/S5Clzlfj4W/4S+qGZKp7sLLOAQxvOorbDqiaINOPADkpu+sNZn6vrQHriCMR48jy74Tmy6Uy8dQoi3oGK3UeQiKAQu3mkKZfB4vn6cdUgCZPEpC740tdkaTSYKeWCwiQS+iHEkmj5Otr0qdN1vfQlifjJ9RQBSfehtNaxBRQBBnT8rlw0z9+n6WzfMSDTsLsS/e1Y5bR2//BXol2hK2hm/387REjMi3ICzmUXNe7oTCyeTPYLGEDP0YirbUdNLEGUKluAlLLEPRFiU6JQ/gIyLObdiLaAG4mVojXDCGiyVolqKLLwXeqWNEzc/MeeVhXDKJbQEZ6gEq5J1YalHS7eL4pi0WINVyNC4k62cpawTrkNpDK3UxEAZ8JOhLBs8gKam1v5efUg1nNLazeKudWkcu5jrFwTIOA/GiY6vUeDK7oUS7+t0ww3bFZLy2IJToStVz24mL7wQ1yjw5orGMiDwVTT4aL848fgK5ehg2XaoF3rDVxDgQjQ9d489LoEGtdF5oG5eegdSLAVD4U1nIzr6kXjlJSkHy1WG04gKqQwhBkU8rzkUQQyPTkuAUrUmwOy49E50sZyGI4OHnGtE1VYvtXqIcSpz9kr8TVTHtdmqfKlfd7ocOZJ9jBmO5JG27tuPcB67bZvfNsH2o90w+EAjkAL8AxwSDwSX1tsiw1SToT5jT8Cdniwn6UcnN5HAdGhunOuVastW1JOiOoBJ3sj+sJoFXf0VEnNm4F9HY6EpczHPS67cyKiQkr8XWvZC6EA9fI1nlFI/znwkRRba6Ejd/AHE0XgSVdYa2ViFYV0cikZUMmXSEWOEnxiFIVmETICQuBeElQV9EstaRxkeM/bFFD7L0vQDE2IewGI4tegIJbN0Bt/4NF9OQVNWzcafKCkgpufTxC7n08Qu36t4YGp56iXwgENgXeAFMyEWTQAiq3TMKL18RZ2+K5Vgy1b/xMAkXK9CsSLoFLJxfgQQagdK5Gzl4y0fodeSqM3BaJyrK5DMk6EmGegqLeUTFacTkwRsYu5ZsdTMWSwmL04jIiwAI6/NwMxFNDrnZQ7Ai05NRS5HkyESdNYeqcxg8QBZQlHrfQSb3cEReoLBFgEpxY9raiRJdKJXP41VjsEUPouJkEBbFejCCCIr2NRKm3ETEeUT1ceSqE5I2eIky1FkTMDRL6juTvxi4HHhjG9hi2AY4AlK12Grj5UsSug8eJqberxYVC8hEU4wggY//EdOHbzCtvKXj1V8m0/qdWbNPvYqmDV5GI4ji1jMoU5ko0Q0l0qt3Zql7cTEdgU2GfhmlOhITR4DwEudAAIQQKHKoWS5i/Rm8845EYCcDF3uhaZ2KmHKE30Ul1+BiIV4+xhH6OBn6CTz6SyeWvQa26EPISo/w0SKvznK+zrZ8yuQreNUH+LO6UBkyMenNmXr55IPB4EXBYPCnbWWMof7Yomeab1fjIUdfxfp1aJx9YkBx0l+rcDEbt/6qAa1tYHQct/4Bt55YZwVDwWqq7pMTcVSAi1k1fNhRcvSV5KoTybKvpZU9jFb2Kbj0lKRLxk7tl6XvJEdd4sSO1zRBFFAhbkWRkeZDrwpt1Lio/qwEmgI8/EjNEtIaN0q2pdK6mRLxLmAli4BFcTGz1jm3BlvsTMi6EZn1DxCeeh/P0HgIvQ3KdQYCgSXAQZvpk+8OLK73SQ11ohOL0OtOStWcF/kvoosugFo+3dolyQDwD0O2unN7m9ngaK3RRWdCYo7zhu9oZKt7UttV6EMou4Oqap+490DkvYgOj4Lyuzd+cJEBvlMh/Np6G/yIvKcR3kE17IghhMexp/hyiI2j9ucgAK8jrvlvQNGw9Hrvrn6I1v9LHsdGrzkQlFOzBqsLsqAFP6gNAD2gVgegDdJo38vXratAqQ0/YAoKslmzprwBLao/TcPmAqR4Hxd/kBC7okp7kMXAZMy8Q5wAbhakZp7V/l8XoUhr4tFJ2PRCUIHFXGy6b7CiYWOwyfusNX79NF79OQl6UyHvQFJKrpqZmpXr8Iesid4IxBGUkqU+xpMKH3QTShxMZG2YHDWqzi6fNRdHtQ5B+K31Yt1BE6aiJEhU9kfoCvLlcHRiNjbdKZMvoLmXfAbW8TlYhDiPCCdCSRtcYiRZ+k7AJiSGE1NHw9ooJK9F8jJ+XgBchPQl6G34O9g0fqe3jJZqs5SC1q23vPT3jul8beEo0R6hx5OjRmDTiQpxK1pn4uVLBDZuguuNcGFTgCKXDP086JeJsSduZuGIYHIRcnPT7BsBrTXRUBRfpg8P3+HX/0MQRrKODPUkIXkFVd5JjUDRFosF5KiLEURR5CUjXaKARUL0xq9exMXs9b7zVL9Kj4Kpu7qnX79JlFPx6vcgMS8ZxvonfvUqIflPwAOpKBbHZy8I4ec1fHo0JfpdEmIfSqwxG7x2JTpQad1aj7u3adb8uZZRI0cjLcnJ1x2HN8OLkAJ/lkmQa+oYkW+BWDpIpn4YQRTJX2Tpu0nQj+pIjHQHgcaNoBIXhVQ1+fYwAaddoDPGp16jwvrvtjdWV5KtbsDFDOIMokLeWXd6vK7EzRQUHSBZu0TqP8lWVyPUX3z8UhtevLMNgX135c53O4FVda0xLFaAyKBMPEqWvgWBopLryFCPJhdaQVBGlIMQJIjKo0gwgGxGpN2pBF1xsSJt5u2co4aZrD/r12TYD+DjY6pK+FZFniME5fJhstS/gQSV4mYy9X8RxBHEUZTgYg6JRq7VomzF7cfcS8nqUoQQ/Dr6N6KVETQw9KwhnHrjSWTnm+YyTZVtkgwVDAa7mxj5poNkVTI8EgQ2FiuJySGQlpxTU6TCyUzKmiiqk2HcTqidDpFjX0S+vTc59rkIveVfiS09D7f+EXQl6AQZ6nHc/IqkEg9f4VOv1x6kKshVJ5ClbqKVOtfxnwNZ6nYslmDJMEeduZzAgHIWTVvC+M/z0fhRZKHxEREnkGnfT5a+Ack6BMVkc30yQ7jqHkSJiyFUyHuJi4NwQhtDaffEYnXqnlRRM4nJaa3oSb2nEUTE8fgYTXWhN4GigLB0MonjYl+Kra8ptr4jJg9Ptr+r/uych9r2JxFL8NTw5xne/2qeueJFEvHqbyYVJZWUF1WglUbZisqSShJxGztu882r33PNwJtYt7KoQew0bDkm47UFEmcvNPkoMtH4CInznTA6Lq8zbG79mahOfsGrehDYdCAkL8On301GmzhZmHUK8kbwqk9opc4lW11Hnvo/8tSR+PiAmrNlL7Xbw2Xp25AUIYk67pSK55P2labyebUWZObYoBXhcA4lchQV8l5K5Htk6Ofw8hGS0hpRKrpGowsARaa+h3y1H9n2cISuQNEpKdwyGdgYAdwoPGhcaREvNrlEOZJS8RZRjibKUZSI0WgKSP820J8S+ekG1zjK5QPEGEyC3pTL+1CiYUT+8+e/ZtLYKZSsKuXX0ZP5+pVvU9uy87No27XNBsdGQ1EmjZnSEGYatgLjrmmJiAxK5Du4mYqiHbbYGYCIdR7a9pDJw3UOcyqi7IHCj4dfku9KopwAIhOhIlTPZG0E4TqPsyH8+qUamZthIFyH6yi/1jgPv6YvakofaIgxBIvFaAWFS9zM+DWTnfuWMej4bmjRijiDQGssFqZyBKrcKXWVW66yx8V0cvQ5yOTMXSeXXkXy50pxKxZL8OtXkz58HyFxDTF5NACVVEcnxfTQ5HUXIYUmJK4EYW3wHmnRmgpr5GbczW3LupVFxCOOOykWjVO0sji1TQjBZU9dzO3H3IsdX79GDlguF2061/7cDE0DI/ItFeEnzv7Vr7XCo7/CxWzAhyBSa1Yf5WAq5R3kqyFpM9yE6AtARJyCV7+HpBJNJhF51haZpGiFTGbb1hZ3K+laORKpF6NED4QuxqO/R5GLJJKatZNYiEtOxcUsAKSEnfpE+d/UWfiyfITkdGJVZZGFIM5euJlUp7jX7I6VunVoLFbW2FckqzguJcahxMShQAKp/8LN70Q5mJg4ss5r1iKbEvkeFovIa7MLiXXbqRxvPTn03IP46d3xIAVoGPr3A9K277R7d/odsBtzJsxDK8VxVx7FsrkrWDx9CYNO2o+9jjQ13psqRuR3EDLUY0nXSAzWy3xNtUYW56V6hKYn0K8lxz4Piz+TfmobiCJZu0U+4wpxF7n6dKoakmv8QAUJ+lDBv8jmZjL1owhtU8HVZPIiUIFTTbM6HxRieNRXybr4SQRkZms0OvXNpYpy+Sh56khIumuqSC/o1TY5Kw9j0x0Ai0WAQpFLmXx1vUxgz+ZHtAgvNr0RMhtovNC+SGWE1UvX0q57AW6fmwmjJlG8qoSBx+9D50AnHp5wHyuCK+nSuxM5bdLLBwshuO7NESyevhR/to+OuzSMG8lQf4zIt1CELiJDPQfECMsL8fAN1WVkBRo3JBuHVEl6jr6cUl5H0T7ZIxQUufj1s0mBrznfTeDSUzYcVllHlyAlu1OiP8Wjv0bRlpg4DEiAcOPWE5BqVaooVhYPorGQ1JW9KZCUoOiMYEHSFWMRY3ci8szavXCFlzL5ElnqWiyWJo/g/EvQnVL5PggJ2omZ1+QjqMSn30ToKGF5VrMt9TD+o4mMfuIzcgpyWDxtCbat8Pg89D2gN79++htaKUaNHE1Gjp+iwmL2PmpPrnr+sjqPJaVk5z16NPAVGOpL8/zNNWySHPWPpKApPOpn4uyRXGiMAR7KuR0f7+BhJo5/XQOVePmCEvk+XjWKDJ5FUgaU1Er0ATcJ+uNTb+DR44izF2F5GZLV5KhLkKzEpiuV4mYScp+UXUq0IyL+XsNS50GgaEN10S7nnfX96NU/azz8RIW4Ea/+EkkxIXk5cVHDPbUeTnGuY8jQzyFIJB9XFhFxhiPwAMKNpk3yPNmERd1i11xYPncFL1zzKrFwerZzNBTll1G/pr7KhMvDhMudh+v0b2cyaewU9jtu74Y217CdMCLfEtEKi8U1Zt6VhMQV2LoTFiuIyDNJiAHE9aG09j6KDr+L0/9T49Zf49a/YNOOmiVwnaibDDS5xNmbmDw0mWj0LIIILuajdQ6WnotMxttbLCNHX0WFurPOXqI1sdmJmm6idLeKH8fNpKkuvpZAUka59eQW3Bg31QFlkhhDW3RD6DV/rkVatQPohBToRN3Z5lorYpH1S2AYmjMmhLIlIiRx9kTjTcW4K9GJsDWCCutBEuyOW/+IV38EGefXGKhxsQg3M/DyzXoHlZTLhymRH1Fp3Upc7I+lF1GVWi/4zD/xAAAgAElEQVSIYOlg6mHhvOf0nfLqTzdqrktNJF8NBFSNGPMqi3xExFkUyQmUi4fR+EBkoPEQEwds+KB1EBEnkyCARmDTg0p50xaNb24E9utJRnbtjNQOO7Unr33dZaUTMZtvX/+BsrVl29s8QwNhZvItlHL5OD79CRAjKk5IC9vz6+fw6zfRKFj3NFUp+emNKqrRQJSjSIh9qElMHIxfv4FCI4CYPA6bdrjVrzgLpqDwpqJzNkS2viEti9SmCzGOwmIBCdGbiDgbhEVcHECpfom87OWUlvfZ8hhykeGU4dX2RsMYWwoZ2X4e+OEu7jvtvyyatgQAj9/D7WNuIR6N8/lzX7FqyWp+/2oadkIhpcRO2Cycuog3bn2Hy5++pHEvwLBNMCLfUhE+IuL0Ojd59ehklitQY2EzPYxwvTGMQdtZhOR1qSYTttiVEvkWbj2VhOiNLQIAFMtv8KoP8fA9CbE7YXHRxk1dr0JmVBxLRNbdecgWvRD+vVEV9YhSaaECHw1F+eTxsRQXlnDkJYfRtU8XMltlcPeXtzLtmz9Ys3wdg08ZiD/LB8AZ/zkVgOJVxfzy0a988MDH2AkbO6Eo/mv9HrCG5ooR+R0QmwCS1Un/tqpzn5qLnVX/+3gPt5qKohUhMZwM/RwWi4iIE7HFCdWDhZuoNYwowzbLnjCn4uft5CsvEep+OBk2zpOXPc8f380gHk3w6+jfeHTS/alQyAGH9q+1v7IVd55wP/MmLwAgIycDvyVRSnHStcc1qO2G7YcR+R0Ml56BoJA6a8lTewafjlPOQAOt9HScqJwEGfo1bN2fuNh3q2wKW9cSVwcjWUFcHAjCFLvaGuZPXkA8mnS9ScHKBX/VinevyeinPmfepAWp16GyEP8ZdSPtd2pHXrsduxVkS8IsvO5I6DjZ6nLczAP0ehmnteW9rlYW1f9Hk4usDlKvrpdpCbkHMXkMWmTX6zg7Mrsf0g+P34O0JJZL0qV3543uv2Leylrv9dx7FyPwLQwzk9+BcEIiq6JhkiIu21KhLichepKrzwViteLSofZCrKIzkrXJcgSZWxzpYtj2XPLI+fTcexdKVpdywOmDyGyVsdH9e/Tvxs/vT0i9zm+fh+VqmesVOzJG5HcgNK2Isy9uppJqVKHWkskTFIsxlInn8KkXcDMbqCDO31B0QjIPF38iKMeReLcTzih8WKwgTl8QmY13YQYALJfFIecctNn7d+rZEY/fk0qWym5tvkW1RIzI70gIQbl8BLf+hWz9z1TJXUEZWeo/SIoJy/OoEAPrHO7WP+LSC4iJIdhiF8CZ0RuaJ32H7Eafwb2Z8f1MXF4359//900PMjQ7jMjvaAiLuBhCwh6QbG3nRLl7+B5BHJeaQYy9EUhC8orqYl86RII9iEvjlmkpSEty/ZsjKFtXTka2H5fHyEFLxHyqOyhl8kl8+gMyM0BXvoVkTXJLDA/jAchR0yiWX+LVn5CpHwY0IXExEbnxuHdD8yLHuGlaNCa6ZkdF+InIs5HZw4mK45Pt8nxU1ah0/lWQoUbW6DuaIEO/sFVt/wwNT6QySjxaVxVPw46EEXkDYXEp5fJ+QuJa4uybrHkDApVsQF0zYaqu1huGxiART7B01jJK19SuM/P+A6O4uOcVXLTr5Uz4ZFIjWGdoKhh3TRNCa83PH0xg7sR57HXknuxRR5bidkEI4gwGAVF9LH71FH7exak3H0fRCicaR1EprkEnk5XcegI+9RY23QnJy0HULoZl2D7EwjFuO/oe/lqyGq00Fz9yPihNp54daFWQw5inPk+WKIAXrnmVgcfvs+mDGlokRuSbEN+//ROv/ettYuEY4z+cyPVvjKDPkN6bHrgtEW4i8hx86mOcfq5uohxDSI5IbnfiqKVeQra6LtlGcApClVBp3d2wtu7ATP9uJquWrCZa6eQ9PDP8eTx+D0pp9jthHxJ19GI17JjUS+QDgcCZwL9xCnU/GgwGn9omVu2gzPhhVipmOR6NE5w8PyXyX7zwDZ88Poa89nmMeOEy2nVvu93s0CKfUvkmXv0VNu2JiSOqG2skcbEIjZV03sRwMwOPGoMWOcQZkipiZtg+ZOVmomtkqmkcHzzAT++OR6tkWxSX5NLHLmgcIw1Ngq32yQcCgU7APcBgYABwSSAQ2G1bGbYjssdhu+PxewBwe93sNqgXAMtmL+edez6gdHUZS2cu4+nhL2x3W5ToTFheQEwelRJ4t55Irn0sufZxaC0BC40HjQ9BOVn6frLVLWSoh7a7fTsaSimmfj2dSWOnkIgl6DWwJ4edPxRflo+CLm1we50OW9KSKYEH2Pe4vdnnmL0ay2xDE6A+M/lDgW+DwWARQCAQ+AA4BbhzWxi2IzLk1P3xZXqZN3khAw7pR699ewJQvq4cKR2h1UpTsrq04Y3TMbLVtakWfVn6P5SK9/DwPRpFhn4Wkcyi9TKGEDc0vI0tmGeueJEpX0wFoOtuXbht9M2ceetpnHnraQB89co4vnvjR/I65DHr5znYcRuXx2V88QaE1nVXI9wUgUDgZiAzGAz+O/n6ImCfYDC4qU4D3YHFW3XSHZRYNM6I/f/FigWFqITin89dwqF/P3C7nlPba9EVI0GHEFlXgGyLXr0fpIqSSUS7PxDCg1ZF6DVDQYcBC1x9kG0+2K727UgopTjCMyw1Q/f43Lw67wkKOreuc/85v85n2rcz6b3frgwYuvGGLYZmSQ9gyebuXJ+ZvKR2n4m6i5PXwbp1FSi14QdMQUE2a9Y0r3js7WnzrZ/ezKLpS2hVkEO77m232Xk2ZHMr+xwsFgMKFfmZYvk5mRyBl3GAJsJxhNZGcdr/uXGJx8nQz6DIo1LdgN6On11z/N2A+tmd3yGPopXFaK2RLouYgjVrypn18xxeuv51pCUZetYQKktD7H5wPw696FCAet+n5nivW6rNUgpat97yMtz1EfnlwJAar9sDtWuXGrYJLo+Lnnvv0mDncxqBV0VohHHryVTK24kwDBDYBNL2T4g9KbO2/1rBjsTKBYVopenUsyNHXnIY79z9AQjB2XedgcfvIR6N89+zHycachZc37r9PQA+e+ZLbvng+gb9fTE0Xeoj8t8AtwcCgQKgEjgZME0hWwhx9sfFpGRp4gTZ+ibKxb1OUw/DdufN29/hm1e+AyEYeMI+/PLRxFRY5Hv3fsjQM4cQDcdIxBO1xsZjCWb9NIeee+/CrJ/nMPmz3+m5184MPHFfhIl62uHY6uiaYDC4AvgX8B0wDXg7GAya1LoWQrl8iDhDAStZ4iCCT73T2GbtEMSjcb54/htikTixcIyf3vuFmlnGobIwsXCMb1//nrZdC/D4Pbg8LqSVjILyutl1r52ZN3kBD531GF+9NI4XrnmVca9/3zgXZGhU6hUnHwwG34ZUc05DS0K4icqj8agfgDAaDzY7N7ZVOwSWy8LtcRFN5kxIl2T3g/sy44fZKKU48epjeeziZ5j542zisTj+LB/XvXEVq5asJjhhHvscuzd9D9iN0U86Wa8A0XCMaeNmcOi5Qxvz0gyNgMl4NWyQuNifkBiOV39Kgt0IySs2a5xHfU6GfhxNDhXyPmyx03a2tGWQiCeIVETIysvimteu5JkrX0JrxSWPXMCAQ/oxd+I8FkxZRLe+Xfj08bHVxce004T7kLMP4pCzD0odL7DvrlhuCzth483wsPvB/RrnwgyNihF5w0aJyDOJcOZm7y/1arL0XQiiaFaTra6lxBq1HS1svmitmfbNH5SsKiW/Uz6PX/wMsUiMfgfsxnVvjODpP0am9o2Gojxz5YuUr6sAILddK+yVxSTiCaTLov1O7Wodv+feu3D9GyOY/NkUdvnbzgw6eb8GuzZD08GIvGGbIiihaqnHKXlQ3Kj2NGXevfdDvnxxHFprp5hYcmF17sR5TP9uZlqBusV/LKWyuDIVSRMuD3PStcdRvq6cwy44mIzsuovD9RnSu+HrHxmaFEbkmxhKKRZOXczYZ74kr10rTr3xRDJyNt6QOVIZ5Yf//YSyFQeeOWSDf/ANgc3OJNgNK9l1KiwubDRbmjo/vjs+JdpCVi+sapzyBPFonAW/L6JVQQ4FXdpg204ainRJOvXsyPEjjq4eozWfPfsVk8b8xm6DenHqjSemFmINOzZG5JsQv46ezFOXv0Ai6oTFuTwuVs7/i5vfu3aj4+499SGWzvoTtOb7//3M/d/d0XihcsKiTD6Ni5locjboj58zIcgb/3kHb4aHix4+j067dmhgQxufrr07U15UgR23kS6J2+0iUhmlc8+O7DygO7ceeTerlqxB2Tbn3nsW17x6BR8+9An5HfM5796z0o716+jf+ODBUURDMZbO+pPM3AyOGX5kI12ZoSlhRL6JoLXm2StfSgk8QCKWYPEfSzY6Lh6Ns3Dq4lTKe+HCvygvqtislm4LpixkCZpOfbulClxtE4SLBAM2uDlcEebBMx91ZrECHhg2ksen7HhFza549h+8ctObLJn1JwJN4cJVACyZuZQ7j7+fNcvXpUoJv//AKJ6aPpL+B9VdpmDFvJXEws5CbCwcY8mMZQ1zEYYmjxH5JsT6ZR48fg97HbnnRse4PC7adm3Dmj/XobUmOz+LrNzMDe6/bkUR4z+cwLwpC5nx/SxcbosOO3fgjrG3YLmsrbLboz4lUz+OJpNyeT+22LgPuKKoEq2SFTA0FP9VslXnbY4Uryph8tgptO6Yz26De7Pg90WUrilNCTSAHVcULvwLaVV/HqWry3j4nCe45rUrmDx2Cu/e+xF57XMZ/tTFtO6Yz95H7smYp74ANFrDgcMGN8LVGZoiRuS3gkQ8wXMjXmb6tzPouc+uXPnsP/BmeDd7vLIV5UUVZLfOSlWXFEJw/v1/5+Ub30ApxW6DejHk1P0ZdJITEbFkxlJCZWEnLK6GGAshuPWTm/jwv59iJ2xOuva4DfpiK0oqufmQ2wiXh7ETjsjGI3FWzFvJ8uAKuvXpusX3Qug1ZOn7EMSAYrLVdZRYYzc6pnXnfHrs3p2lM5ehNTtM1Ed5cQU3Db2NSEUEaUn2OmpPKooq0gS+CttW+LJ9qTBJrXUqe/XxS55FK81fi1Zx/+kjeeinu+napwv3fnMbcyfMo8eA7nTvu+WfpaFlYkR+K/juzR+ZPPZ3YpEYM76fyadPfMapN564WWPXrSji1qPupryogoKubbhj7L9SM++DzhzCfsfvjbJV2mLrh//9hNFPfo6Ugu79uvHvUTekHg4Aee3zuOi/527y3EtnLsOO2ymBr0IrRauCVptl//pIKqiZOC1wiixprTe4LiCl5F8fXM8f38/E6/ey2+BeadtDZSFevfkt/lq8mmMuP4J9jv7bVtnW1Jj763wSsXhKuGf/PCe1mCoECCmRliQRS4CGyuJQ2nitNQumLEyrF1+48K/Uzx12bk+Hnds3wJUYmhNm+X0rKF1TlqoZEo8ltqi++8ePjaF0TSmJWII1y9by7Rs/pLYlYgk8fk+taJqxT39BLBwjUhll8fQlrJxXuFV2d9ilPZFkNEcVQgo67NweX+bmfxOpiU134uyFxs+awkzef+koLt1tBOd0voS37ngvtV95UUWq6xU4bqY9Dx9AnyG9az0MnhvxMhM/mcyCKQt5+vIXWB5csVW2NTU6BzqmHrAuj4ud99yJq56/lO79ulLQrYBe+wfYfQOlgYUUHDP8CMcnX+N2KaVazP0xbB+MyG8FB54xmIwcP/5sP/4sP0defNhmj7VcVqo1nhBguR3Xyzv3fMB53S/lwp0vZ8YPs9LG1JxlK63J3oxF1brwZ/lqvaeVZsX8Qj5+dMxWHRMhKJePMH3OvVx0wG68ctt8ytaVYydsvn7lWxZNW8zTw59neP+ruaTXVUz/dsYmD/nn3BWph6i0JKsWr07bHglF+fHd8fwy6tdU2n5zoEOPdlz3+lX0P6gvB54xmEsfv5A9Dx9Al96dKS4sYfZPc5j23Yw0Ea9ij8N255QbTmDFvJVpBb4FMPrJzxvsGgzNDyPyW0FBlzY8OukBbvzf1Tw66QE69+q02WNPvPpY8tvnAc7X60POPpC/Fq/i8+e/RtnKyWy84sW0Mde+fiXd+3ejbbcCLn/6EloV5Gzw+MpWzPklyILfF7F+Qxhfpo82nVo7PvsaQmLHbcrW1aP+tpB88+ZSIpWxNFeClIIls/5k8udTseM2sUiMp694EWVvvO3AoecehDfDgzfDi9fvIbBfz+rrU4qrh/yHV256gxeueZXHLnp66+1uBLr07sxpt5zEOXedkcpnWPj7opQLx+1xc/otJ9Old/rv1Jxf5mInbD546JO096VlbXChvbyogqWzljnuH8MOi/HJbyUZORlbVa87+Os8ytaV4/a6WLVkDUWFTkaoqKG664tg50An7vnqVt687R1evvENvn39e658/jIyW6W7dbTW/Pecx5k7IYjWmqFnHcg5d5+R2i6E4PYxNzPqkTFEKsJM+3YmKm6DgKMv/b8tvpaaFHQtwONzE4s4YmW5LHoNDNCjf1eo8bApLypn5HlPcN0bIzZ4rKMu/T+69+vG2uXrGHBIvzQRK1pZzLI5y1PnmfLltI36/5sCiXiCdSuKWDp1Ibed8CC2rVC2Yte9dubql69gyGmD+PjRMSjbRlqSg4YN5qAzh3BZn3+mjmHHbYpWFpOVm0motNpX361vF0667vha55z76zweGPYIQkBe+1zu/upW/FmNlyRnaDyMyDcw37/9U8o3bbkU4z+cyKk3nsiBwwbx7Vs/Ii3JhQ/XXkSd/NnvfPvGj0RDUWaNn8vbd77HxQ+fl7ZPUWExs36ak5oVfvXyOM6+a1iaAOa1z+OCB84GIFIZIbyuDG9u1gazamOROC9d/xpzJ8xj/5P25bSbT6pTUI+4+FAKF/zFzB9n0Wdwb064+lgKurZBCMGQYYOc2ugAGqaNm0GkMrrRdYCqJubrk9MmB3fyYSKkoG23giYt8BXFFfz7/+6iZHUpdjyRtug9/7eFvHzj61z94jC69G7L6qUl7HXkniybs5zv3vqR7NZZqVo1ltti+bwVHDBsED+99wvh8gjD/n0yB50xpM7zvn//qFQ2bVFhCZM/+50DThu0/S/Y0OQwIt/A7DSgB7PGzyUeiWMnFKNGjqZsbTkXPnQOp91yMm6PC4/fU2tc6epSVDK23JnVFdXaJ7NVRlp6fHZ+1kYF0Jfpo0v3glTbsRk/zGL8hxPpsXt3Djt/KLN/nss7937IoqlOS95PHhuLL8vP8VcdlTpGIpZg6jd/4PG5ufC/56RF/VRx3t1n8MtH4wmVOg+3jBw/Hv/WJV95fG5Gfn8nT1/7Gh6fh7/fcfpWHaeh+Ob171nz59o0N1YVWmlKC38nX7/JIYd4KFFP8sNHc3np+terK0wmiVRGeeyiZxBCkJHjZ+TE+zZaviI7PwvpkqjkQyWz1YZzJwwtGyPyW8Dkz35nwZSFDDi0P70HBjY9oA5OuPoYVi1dzfgPJqbeG/f69xxyzoF079dtg+P2PXYvRo0cTdQdRdmK40ccU2sfX6aPs247jdf+/TZa6S2KP180bTEPn/sEsXCMiZ9OZsGURUz+bEpaRAzAT++PT4m81pp7TnkoFe++z9F/47InL0o/sNbkiBt56P35PPOfAhStOee+G+p8GGwuO/Xvxo1vX73V4xuSyWOmpAm8P9tHuDyCEAKXR3D2tX8iiJOIxRkx5BHWLJe11lLAuddVn4W0BPN/W7jBSByAc+89izXL1lK48C8GnTKQPQ/ffdtfnKFZYER+M/np/V94+frXiYZjfPnSOG55/7qN+uQnfjqZd+7+gKz8LC5/6uJU/LLL7eKA0weliTzAr2OmbFTkc9rk8PCEe1kyYxnturclv0Nenft9/txXqdnbN69+x6HnDaV9j9plaNenZmmEWDjGzB9n1RJ4gB41kmyKCotZNG1Jatb584cTuPSJC9O+PQiK8DCeXfrGeXhUCZqFFMmOxKNxVi9bS5tO+VuUSNbcCJWHUz8LAeGKCOCI9oDBMZQN336Uy69fZ7P6T0Fa6EwSaUl8WT6ioWgqz6Fd94KNnjevXS73fH3bNr0WQ/PEiPxmMuWLqalOPfFonNk/z9mgyBcVFvPMlS8Sj8RZvWwNI897kod+uju1vc+g3mTlZVJRXAk4M7MugY51HuvjR8fw8SNjyMrLZMSLw1n8x1KmfzuDg/9+IG271f5DD5VHUj8LKQnXeL0xeu3XM+Xq8fjcTmhnUnOEFHh8bnbZcyfOf+BslFK8ces7TBr9WyqEUQhBfoe8Wu4hTQZgAcnMTbIpXev4qStKKnF7XNwx9l8tNonnwGGD+fiRMU4pYdtO0/DfvnPzx4ROoGyikbq/2Vgui/5D+3LxyPN46/Z3WbNsLcddddRmPbgNBjAiv9n0O7AP08fNIBqOOT00kwIfKgvx/oMfU7amjHNuPYVWndpQtq682h2hnXolNZGW5MlpD/Pida+zbPYyBp08kIEn7lvrnCvmreTjR8YQi8QoKoxx32kPY9s2iViCca//wGOTH6i1YHrmbafy0nWvI6Sg98CedOvbZbOur0vvzlz53KV8+vhYChf9RdHK6jrwR1xyGGffMSz1+ucPJvD9mz8SDcew3Ba5bVrRcZcOXPDg2WnH1FqzbPYaynL/Q5f2zwFuKuTt/PDO+ORCpE0sFOPTxz/jH49dsFl2NgcS8QS/fzkNIQX9h/Zl1MjRqQdmOoJoSOM8BNPpf1AfFk1fQrd+3bjsyYvIys3k8qcvAZxJxtSvp+PP8RPYZ9cmvfBsaHyMyG8mB599IJbHIjhxPvsc/Tf6DHaKcD1y/lMEJ80nEUswfdwMHp38AF16daJLr04sD65EKcUxw4+odTyPz8PwJy9i1ZLVzP55LounL2GnAT3S9olURtMWUqOhSCoaUdmKFfMK2XWv9L6rB5w2iH4H7EZlaYiOu3bg5w8mMGHUJAL77spxVx61wbo2kcoIz1/9ChXF5ej1wtgriirSXhf/VUIi2eDCjtsE9u3JiBcuS9tHa80Tlz7H719OA6057qrzOena4wDwZRYiLZkqsevPqZ2k1RyxEzbjP5rIx4+OobiwGIQgt20r4rF4LYFv3TGPrPxs/pyzPC1kVlqSvx0xgKtfrrvVorIVdxx3HyvnF6I1HH7BwZzxn1O352UZmjlG5DcTIQQHDRvCQcPSQ9YW/7GkOtlEwKola9h5jx7c+ulNBCfOJzM3Y4O+9pULCvn34Xemqk9e+dyl/O3/qkv09ti9G733DzD75zlo7aTFLw+uIB5LIIRTpqAuCheu4s85y5k/ZRGv3fIWsXCMOROCWC6LY69waozPm7yAhb8vYsjxeyH8Pl675S1CpZW1BB5gz8PTywYPPGEfPn3iM+LROPFInJXzC1m5oJCOu1TXhC8qLGbK578TT5ZOHvXI6JTIDz3rAKaNm8HMH2fTvW9XTrq2dpx3c+S5q15i0mdT0gqOrV6yBo/Pk76+IWDwaftz+PkH88pNb1K2rpx9jtmLBVMW0nHXjpxQoxnI+qyYt5IV8wpT4ZFfvfytEXnDRqm3yAcCgbsAOxgM3l5/c5ofex4+gEljp2AnbHwZXjr1dHzrLrdrk23Xfv9qOvFYItX27bs3f0gTeSkl1785gr8WryazVQa+DC9jnvmC1UvXULjgL/59+B0cc/mRHHru0NSYH98bz8s3vIFWOtVWDpzF1Pm/LQCcBKIn/vEsyla8d/8o2nYvYOW8lbUKlwEceMYg9j12rxo2T2PauBmcesPxvHn7e2it+XPuch484xEenfxgaj9fpo+aabUZOdXhfm6vmxveqk702RBaa4KT5mPHbHrvH2jynY4mf/H/7d1nYBRVF4Dhd2ZLKgmQhN7bhCIQKUECilJULBQRkS6iWD4VFUVEBWwo0rErKipKUUR6ERSQXkKHgQABAgGSEAhp278fG4YsScgSkmyy3ucX7M7unAzh7N07954T7VpRUoLA8gHcP6wLG+dv5szRrJpDDlg0bRlb/tzOszOG3tSmuuCwIG31jSRJhFTJ/Qa8IFxV4CSvKEowMBl4HJiQz+Fea9i0ITRqF05qchrdn+mMVXadX81MyyR69V4CywXQ5M5GLvOnVRtUQW/QYbPYMPoZqdU054hfkiQqZ2vS3POVhxn/2CSO7T6B3Wrn5zFzqXd7He3bwvo5G7VRo96oR6eTQQeyXkfUI22IXr2Hz//3jXaMzWoj7vAZl2V+ZUICMfoYuWfAXXR/+dpSzT1r9zH96S8xZ92X0ORy3yEg2J9nPx3KrDdnY/Q18PwXw2720jLztR/Z+PsWJAkatg1nxE8vuly/A/8eYsGkxYRULkv/9x53q1FKUarZqDox0cexW+1IEtRtUZd2vdqwfu4mKtSqwLnYC9oHusPh4PyJC0zoO4VvjnyaY17dbrdr94Bu79Ico6/zegeFBjF85nP8+t58AoIDeHrqE8X+cwqly62M5LsBR4FJhRRLiWQ1W1k0Yynnjp2n4+C7UVrXd3lep9dpuw7LhZXRNhZdfe1b975H0tmL4ICOgzrQf+y1zTsRnZry+DuPsvG3LTRoXZ/uw13XvmemmZg0aDpHtsXQoHU9Xp31Ir4BPiSeTtSWSco6mYvxyVqSr3d7HY5sj8FqtiLLEkMmDOJKcip1mtaiUp0KvNxmlMvUwdW58ezSr2RgybQQ0aUZFpOVuMNxhFYL4eDGw9prLSYLQaFlMGeYcTjg7v535rh2bR5uRZuHW930NQfn3PPfs9drHz771h3gckIKZSs4i7VdjE/mk/7TMGeY0el1XDx3ibcWvF6gcxWWV398kXEPj+fc8fPIOpmEkwnMHjsPS6YFnV7nXFF1Kc3lemekZmKz2tAbXP8rfj38e7Yu2QFA1fpVeHf5aO1mfvOOTWnesSmC4I4CJ3lVVX8EUBRlbEFeHxISmO8xYWGeHZkBTH3mK9vxK3MAACAASURBVP76ybmSZPvyaL7eM5Eq1y33O7YnlvMnE/Dv0Ngl5iM7j5Ecn6y1cFv3ywZe/sx1s1C/17vT7/XuuZ579terOLLtKBaTlSPbjrL+53/o91Yv+r/di+nPfoOs11E2LIg7u7XU6pLUalgFu92OJEtIskSHRyIJyVpTr+44hpztRq6sk7QPi+xsZhs2s42/Z/1N9Np9JJ+7DBIMGtsbHz8jpgwzPv5Gnp82hIBgf/ZtOEjckXi2/7mV+5/s6DIqTUtJZ8XMtej0Ou578h583VwT73A4CAopw+WEFAD0Bh016oTh4+d8vflyGnqDDnOG89vI2aNnPf77EhZWxrkzOas2TWaaSbveNqsNSZao07QmR3ce117TMLI+lXOZctn4+xZtqu3MkTPY0zKpWMczyyY9fV0LQsR8Tb5JXlGUR4Ep1z18WFXVTrdy4qSk1Bzt7rILu25U7Cm71uzX1sfLOonoDYcxBF3bIv7Prxv4YdRsZJ1MUPlAPlw7TttuLvv6aE0hZJ2zzsrN/EwJ8Ze0/+g2q42Ec5dJSLhCxP0teG9FJZLOXkSJbEBqhpXUDOf7Lv7qr2yJW2LdH9to/2hbAIIql3d5f71Bj81mdxlZSpKznpjB10Bi/CUS466VT5j7ySIGje/Hz+/MJTM9ky3Lo2nSviELZyzHlG5mx4rdmK3Xdto6HA5GdniH8yfOgySx6qd1jFs62u2ff+SvL/PV8O+xWqwM+WgAKalmSDUTFlaGstXD8Av0w2qxIUkSUb3aevz35Z9fNrhsfnI47ASWC8RitmK32rmSlMrlxBTteb2Pnn7vPZ5r3CHVypNwylkOQZIkrLLskZ+vpPw/vBneGrMsS24Njq+Xb5JXVXU+MP+m39lLtH6wBSu+Xo3Z5FwGV+/2Oi7PL/lshTaFIUkSB/89TMv7IwAIrhDMiB9fZPbYuZw/eYH4Y+eZN34BvUf1dOvc9z7ZkQ3zNmofMkZfA5lpmaQkXuH4nlhqNamRo8hXzcbViTsch8VkxeFwuKx40Rv12G3ZSgHrdNRoVI2YXSe0x5p1bMrpQ3E0aqtQpUEVolfv0Z7LuJLB7LFzSU9xVkHctGArKUlXMKU74zNlmFG3HdWS/JWLqZw7fl5bfRSTVVLX3abhtW6ryfg1Y3N9zjfAh/Frx7J92S7KVgimeSfPT1/MG79Am16SZIn+4/pQu2lNxj00Hju41L7XG3Q8OWEgtfNYeTVq7qt8/8bPZKZm0ndM76wb2YJw80rVEspdq3azYd4m6kbUpusz9xbLaoveo3pStUEVLpxM4I7urXOUE6hYK4zzJ85js9qx22yEVnMdLTe5sxEBZQMwHXLWWl/+9Sqadbwtx9x+bkKrhTBt+wRGd3mX8ycvsGj6MjbM30TqxVQcDudI+dVZL3DbXY211wx8vy+yTub0oTg6D7mHuhGua+/rtXDO2dssNnQGmSZR4S5JPqRKeV77+SUkSSIjNYMFkxZhybQgSRLlKpUl6cy1kb0504zRx4DRz4jVbEFvNGgfcOC8+RoQ7M/lxBRkWSasRqjbCd4dgeUCubtfznsB7khJTCH5/KWsm9+F89+gTPlArUuYw+7gn9kbqHd7HWS9Dq677yHrZOq1qJvb2wBQsVYF3pjzSqHEJfy3lZokf3THMW1lR/RfezFnWOg54uEiP68kSbTrdUeezw+bOoQvX/qO8ycu0Gdkt1zXxF9JunJthCfJWjkDd9hsdu1DBCDxdJLL7snFny7n7NF46rWoS92I2vgG+Nyw3+urP77An9OWkZqcygPP3suFmHiMfs513EY/I+dOnGdA1acILBfIG3NeZsqWj1g3519sVht/TnNt0O2wO9i5cjc9RzyM3WJDadNAK9yWfD6ZiQOmk5qchizLNGoXzqMjezA8ciSJp5No2701z3w69JYKlRXUvnUHmDRoBpIkUal2BcYtHZ1r5c+8pKeks3PlboJCg2jaobF2D+LFb57l9bve1v6tzxyNR5JlIh9syeaFW5FkiTIhQVhMZnq8/DBV61e+0WkEoVDccpIvrvXxx/fEuhTQOrhZxb1Jj6IVFBqkrfnOa16tz1u9mDb0c2SdTKU6FVxG3tc7dTCOr4Z/hyXTzKAP+9EoKpyAsgGkJOZ8X72PnoObDqNuPYIkyzlG9bnxC/Sjz+hHtL/fFlmP0zHn2LVqD9XCq7J+7r/YbXZSElP44oWZfPzPu/R4+SG+G/lTjlU44Py3WDhpMaHVQ4nIqnR4OSGF19q9RXrKtfnps0fjWTBpkTbPvH15NJGr9tDivgiX97Pb7PwxZTGHNqm06dbKZQ9AYfn1vfnaFNv52Avs+XsfrbrmbBZuzrTwx5RFXIhN4N6hnWjQqh6mdBNv3DOGK0lXAIl7h3akz+hegHNJbIWaYVq7QrvdTlBoGZ79dCh9x/Smas0QUtMtOc4jCEWpZO8uyaZxu3BknXPFiI+fkTu6FWxpXl5SElNYOG0JK75ZrXUdKiwRnZsxdfsE3vnzDd5d/pa25jk3H/WZxIk9scSpZ5k4YBqmdDMvf/c/bZWGJEFYjVBqN61JjYbVsFvtWExWzBlmvnxpJgmnE3N938S4JKYN/ZxP+k3l1ME47XFJknjw+ft55883aNsj0mVknX2pZVBIGXT6PEoipJuIU88waeAMwDlStl73gZB2KZ0j22Jc1uNfvdcQs+s4f89eT8KpRJZ9tYrFny7n4MbDzB47l50rovO8VgVVpnygVi7C4ci71vrMET+w7MtVbF64jfG9J5JwKpET+06SlpyGKd2MKd3EounLmD7sS22+/dVZL1CzSQ1Cq4fw7PQnKVexLODcxORXBPPqGakZnDpwWtsBKwjXKzXTNdWUqoxd8ibRq/ZQvWG1HCPAW2E1Wxnd5V0uXbiMrJOJXr2HUfNGFNr7A5StEKyt8c6Lw+EgJeHa6gu7zUHa5TSUyPq8NPN5Fkz6k5CqITw1cRDBFYLZvHAbX700U/tQSj53iYkDpvHxP+/leO/3H5ngHEU7HBzZEcPne6fkmB9v0LoeDduGs3/9QSRZYvBH/bXnHnz+fo7uiGH/+kNIOglZlqndtCYn9p7Ubqxeuej8tpGjoqTknL+/muANPnqq1q9My/tvZ9uSHXz+v2+RJGfVzPDI+tqHiznTwqmDcYX6bw0wdOJgJg6cRsLpJO4ZcBcN2+beG+Dw1qNYsq6trNNx+nAcNRpXx2Jx7Zm6a+VuFk1fxp2PRVFNqZrnzeLCFn/sHO/c/z42mx1ffx8+WP0O5SqJHbCCq1KT5AFqNq5BzcY18j/wJiWeSSI12blJxWaxcXDj4UI/hzskSeKeAXexYf5mJAnqt6yn3eht1fV2WnW93eX4Nt1aEXvgFItnLHPO0TvQpgqys9vtJJxM1LbDmzMspCRdIaSK601iWZYZ8dOLXIxPxj/Iz6UnqG+AD4+N7sWFU1+QlpxOt+EPcP+wLnzcZwpHdsTgsDno9qKz5krdiNoMmTCAld+uoXp4VbYu3qGNNH38fXjpm2dpek8TZFlm7U/rXHboJp+/fO16yBIRXQp/1UxYjdBcPwivF/lQS1Z/vxaLyVkrqE7z2nz23Nc5ujxZMi0smLSIhVOX0GlQB4LDgmhyV2PqNKtV6LFnt+TzFaSlpIPD+SH69+wNWn0gQbiqVCX5olK+UrmsJtRmZJ1cJB8k7nri4wG07dkGi8lC43YNb1hGVpIkmt9zG0tmLMeRdSfWNzDnlIAsy0R0acaBDYcAB5XqVKJcpbLa83v/2c/uNftoeEcDWnVtkSP5XzVp4HSSzznLF8wb/wfte0cx8teXidl1HL9AX6o3rAY4d8PGqWcJCPYnonNTjL4GNszfDEBI1fI0ubORNi1U87YaHNpyxNkO0WLj5IFT2vn0eh0XzyZTq0lN7DY7s0bPZueK3TS9qxGDPxqQ42ZpZpqJ+GPnqFi7wg1b47mrz1u9qNGoOklnLtKmWyv2bziYdQ2v8fE3Yko3O+vFW20s/3o1sl5mweTFvP3HyByrmwpTYNkA9AY9VrMVnV6Xo7G7IIBI8gAY/Yy8u/wtFs1Yhm+ALz1eechjsUiSRHibBm4fbzFZMPoZtLXqZcrnvlli+Mzn2LpoBxaThTu6t9aS7M7Ve5g8aAbmTAurv19L5EMteWzUI4TVCM3xHmmX0rPF6Vw3HxRSJkeBrV/f/401P/6DJdPCke0xjP79NSI6NyPtcjqN2oXz7YhZxB87x4PP30ev17pjMVlRtx4ldt9JyLYB1+GAvf8coG5EHaJX72H9HOeegU0Lt1GuSgiPjuyuXYPVP/zNbxMW4nA40Ol1vLtstMsegYKQZdllZdXssXNdn9fJvPTtc0x+4lOspmtTOHarHYvdwt6/9xdpku82/EFioo9zfNcJmtzZiHsG3FVk5xJKL5Hks1SsVYGnJg32dBg3rXG7htSNqE3MzuMgSQx8v2+ux+kN+hw9Xw9uPMyHj07U6pnbLDY2/7GN3Wv2MenfDwi+7h5C9+EPsHDqUiRZolFUeK6dqQCO7TquzWVLEsQdPqOtZ5/21BfsXBGN1Wzls+e+4f0VbzPwvcex2+0MbzWSpLMXndMhEljMFtbP2ci/8zdxZ58oZ112nHP1idluMH/UZzJHtsW4bDZa/tVqnvxkoFvX8PjuE6hbj6JE1s9R0z+76o2qs2O5643gGo2qc++Qjiz/ZjWSJCHLEhaTFYOPgTrNa7l1/oLyL+PH2wtGFuk5hNJPJPlSTqfXMfr317lwMoEy5QNzdIrKy5WLqUzoN9WlYQU4b/467HZidh3PccOz+8sP0erBlpjSMqnVtGauU0lxh89gzrQg62RknYwkyzRp30h7/tTB09qNWlknE3/8HNXCqyLLMuOWvsmf05diSjdzPvYChzcfITMt07mqCgnfQD9nCWW7nXuHdgScUzTq1qMuP4fOoCMwj2801zu0WeXjx6dgt9mRdTIjf3k5zxux3V96gCWfLtNq5OsMOiwmM3/N+ge71Y7OoCO0aihVlcq0fqglze65za0YBKEoiSTvBSRJomKtCjkez0jNYO/fByhbMTjHDtvs5YavZ7Pa2bJoO4s/XU6nwXe7TFncaAOPOdPCuG7jSbucjiRJ+Af7M27Jmy5TP50GdWDe+AWAs0xDeJtrCdVistJlSEcq163E75/8ybHoE1gynTtpqzeqxuTN4zm5/xRN2zbAYXCuDPLxNxIU6ixkdvWGaMO2Ct1e7JrfZQNg6+IdLtdh6+LtWpLfv/4g+9YdILxNAyI6N8NqsdJx0N1smL8Jm8VGuUplWTh1KbIuqwiZxUbq5VRe+eEFt84tCMVBJHkvZUo3MeqesaQkpuBwOOjxysM8/MK1xHf2agOLqySoHl6NSrUrYLfb2bZ0J5ZMC7H7T1GhZphbjS0uX7iMJdMKWSUXUpOuaFM6e9bu45d35xEQHMCTnwzCarES0amZdg9hweRF/Jk1FXRHj0ieGN+f87EXULcepcV9Edz5WBSyLHPbXY0JzbbpTJIkxiwaxS/j5mG32+n7Tu+bagpe7/Y6rPtlg1ZZs07WHPr+9QeZOHA65gwzK2euYeikQfwybj6XstXNj485R+LpRPQ+Bgw+BmSdXODSyoJQVESSL+XMmRZ0ehmd3rVZScyu41y6cFkbpS77ciUPPncfX7zwLduX7iSgXABGPyOSBEgSvV7rxn1PdUan1/FO1/ezzalLnD0a71aSD6lanrAaoSScTkBConF75+qglMQUpgz5DHOGGUmSuJyYwqSNH2qvs9vt/DFpsTanvvG3zfQZ1VNrXJ2firUqMOC9x9myaDux+05RqU5Ft5tbRz3ShtTkVKL/2kvzjrdpFTv3rTtwbb1+hpn1czeSmpxz17HFZCWyW2vqNqtFUGgQkQ+3zHGMIHiSSPKl2JwPf2PJZyvQ63W88PWzLq0Dy5Qv4zINkXY5nc0Lt7FjeTTmTAu2hCs0bhfOA0M7UrtVfQLLXtv12XFgB05n7YrV6WVu63DjUglXyTqZcctGs2nBFoy+Rtr2jAQgJfGKlnQdDgcXzya7vE6SJIz+RjKyyiBIEhhuopZMStIVRnUcgynNhM6gI3bfSbf7nkqSxH1Pdea+pzq7PH7myNlrP5depmmHJhzefCTX96jeoAr3Dr2lytuCUGRKTVkDwVXCqUSWf7kau9WOOdPCVy/NdHk+sFwAcrYyBHarjfQrGTjsWStprDb0RgORD9xOcnwyFtO1Ug539WnHG/NeZeAHfRmzeBRz3v+NMV0/YOfK3fnG5V/Gj06D7ubOx6K06o5V6lemaoMq+Ab44OPvQ5ch97i8RpIkXv3hBcpWLEtguQCemTHUZZ271WzlwIZDnDpwOtdzHos+gd3mwGqxYUo3s+XPbfnGeT273c7xPbHEHztHanIqe/7erz3nsDu4/+nOPDN9SI59CLJOpkFk/hVFBcFTxEi+lHI4HNn7ZGtVKa8qV6ksjaMacnRHDA6Hc1qibY9Iln6xgssJKciSROTDLXm8+jPY7Q6Cw4J4f9U72oheaV0fpXV9JvSbyr51B7BZbMwY9iXj145z6TnrDlknM2bRG+zfcAj/IL9cyyw3igrn872TczxutVgZ8+AHxMecx2638+jIHgx+p5fLMdUaVMGeNdVj8NFT97qa//lxOBxM7D+NQ5uPYLfb6T78QWRZxsbV93TOt7ft0YYW90bw4aMTObrjGLJO5oHn7nOrbLQgeIpI8qVUhZph3PtkR5Z/tQpZp+Op6xo6S5LE678M58C/hzAYDYTf0QBJkpi44QPij50joGwAI9qNJjM1E4Dkc8ls/mMrnZ9wHWXHqWe06pM6ncyF2As3neTBmSgjCtDYI3bfKeJjzpGZ1UJx0YylOZJ8WI1QRs55heVfraJCrTB6vZZ7O8W8nI05x8FNqja9tXjGMnqP6sHssfMAZ4OPi/HJ+Pj7MGngdE4diqNN99Y8/9lTOe6FCEJJI5J8Kdb3nd70fLUbeoMOvTHnP6VOr6NphyYuj+mNeqo3rMaaH/9xmbO3We345VIKoOPAu1k4ZTEAPgE+1G+Zd6OLohAcGqS1UJQkiXIVcy/AFd6mwU3tFM4uINjfpR5NQNkA4g6f1er2Z6aZWD93I5cTUojZdRybxcaulbtZP3djgZuWCEJxEUm+lLu+/Z+7DL4G9EYd5gxnAi1fpRxte0TmOK7bi12pF1GbpPiLRHRq5vZmq8ISViOUp6c8wbzxCwgODeL5L9xbcXMzylYI5qnJg5k9bh7+Zfx44athbF64TasLozfoKFM+0OVbjd1q19ogCkJJJpL8f1TbHpFsXbSd3Wv2Ual2Rd78bUSe7RQbt29YzNG5iurZhqiebfI/MJt1c/5l65IdNI5qyP3DOnP+xAVO7ImlXou6LuUYzJkWriRdoW3PSJdNX5XqVOTk/lOo22OoWr8yTe5qRIOW9dj91z4cDgd+ZfyIeuQOti/bybnjF2jZ9fYCTWMJQlGTrpafLUa1gBNJSanY7Xmf21s7rpc0oaGBJCamejqMm5Lfdd65cjczhn2ptTTsOKgDa2b9g6yTcdgdjF0yipqNa3D6UBzvdv8Ic6aFynUqMnbJaJdvRtuW7uTz57/RatKMXzuOwHIBXDiVSJW6lVg5cw0LJi3CarZg8DUyYd17hFYLKXDcJZGIuXi4E7MsS4SEBALUBmLdfW+xhPI/zt1NQ6XJib2xmDOvbWTauTwac4aZzNRMTOkmNv2xFYC54xeQdjkdS6aFcycusDnr8avWzPobc4azA5TVYiV69R78g/yp1aQGRj8jmxZswZRucvbfdTg4vCX3dfSC4EkiyQteJ6JTM4y+RnQGHUY/Iw3bKlrteaOfUStBbPQ1ZGurKGHI1pbRnGEm7si10g+SLFPluro9SmQDrZWj3e6gZuPqRfpzCUJBiDl5wevUjajNmEWjOPDvIepG1KZBq3oEBPuzb/1BWnRpTvveztIF/cY8Ruzek5yPTaBxu3DaZOsbvG7uRlIvXpvGqtmoeo4m6f3HPUZAWX9OHzpDp8EdtKYpglCSFDjJK4oSBUwBjEASMERV1ZOFFZgg3IraTWtSu2lN7e/9x/Uh/vh5AK1hSkjV8kze8hF2u92lgTk4dwhf7bYlyRIVa+es8qk36nl0ZI+i+hEEoVDcynTNbGCoqqrNs/48vXBCEoTC9/PYOYy6+x3euPsdfnr7V5fnrk/wAMEVgrW693qjnp4jRO9UoXQqUJJXFMUHeEtV1b1ZD+0FPNcYVRBuwJxhZsU3f2HOtGDJtLDquzXaDtq8/PjWL1qpCEmSXNr7CUJpUqDpGlVVTcDPAIqiyMBYYOHNvEfWUqAbCgsrU4DoPEvEXDxuJmab1YbeoMdsda640Rn0VK5aViuglht9tnIFElA+NLBQrpO3X+uSQsR8Tb5JXlGUR3HOvWd3WFXVToqiGIFZWe/zYY4X34BYJ18y/FdiHj7zOb588TsAhk19guRLGTc8/umpQ5g0aAYWk4VOg+/GPzT4lq/Tf+Vae5q3xpxtnfxNKfBmKEVRAoFFOG+69s8a3bujFmIzVIlR2mJeN+dfDq4/SHhUOB36ts+xzj92/ykm9J3KlYtX6Dqsi9t15XNjt9uxW+251gUqiNJ2rUHEXFyKcjPUrfz2/gzEAM+oqmrP72BBuFVbFm3n+zd+xpxhZtuyXfj4GWl7XbmDL/73rdaib+XMNbTp1tpllc1VDoeDDfM3cTz6BK0fbEmjqPAcx8iyjGwUW0mE0q2gN14jgG5AFLBLUZTdiqIsK9TIBOE6R7bHaJUzTRlmDm89muMYU/q1L5SyLGHKyP0L5sqZa/j+9Z9Y9d1aJvSbyrHoE0UTtCB4WEFvvEbj2rJCEIrc7Z2bsfandVhNFvQ+BlrcF5HjmEEf9GXa0C9w4ECJbECDlrn3pt29eg+mrA8Mu82OuvUodbOaeAuCNxE7XoVSo8mdjXhjziuc2hNL9aY1aXiHkuOYiM7N+HzfZNIupRNaPSTP2jzNOzdD3XYUU7oZWZYJbyO6OwneSSR5oVQJb9OA9g+1uOFNKv8g/3zr3t/7ZEcCywZwbNdxWj/YkjrNxShe8E4iyQv/SZIk0bZnJJXrVsrRnFsQvIlI8sJ/ksPh4JN+0zi48TAWk4XAcgG8/N3/aNg25xSQIJRmIskLpVrqpTRmjpjF2ZhzdBp8N8FhQQSFlMm33+u54+c5uMmZ4AFSk9P4pP9UvjnyqWjOLXgVkeSFUu3bEbPYuSIam8XGD2/8hMHHgCRLPPDsffR6vXuer/MP8uf6jYDmTAsWk1UkecGriJ0eQql29mi81lzb4XAmalO6mdU/rL3h64LDgnhm2hCMvgYkydnYvP1jUQVujC4IJZUYyQulWtdhXfjhzdlIkqRthJJ1kkuz7ry07dGGO7pHcnx3LAB1mtcqwkgFwTNEkhdKtQ5921OjcXUunEzAZrWx5LMVlK0YzFOTBrv1ekmSxCYowauJJC+UenWa1aJOs1oARF1XyyY3x6JP8En/aaRdTuPhF7qK7k6CVxNz8sJ/zmfPfU1KYgo2i42lX6wk7vAZT4ckCEVGJHnBK6Ump7Jv3QGSzl7M8dzVImfgnK4xZfu7IHgbkeQFr2A1W7U170lnL/LKHaOY+uTnjIgazZHtMS7HDvqwLwZfAwYfA7fd1YjazXKWIhYEbyHm5IVSLeF0Iu/1nEDiqUQkWWLAu32wWmxkpGZqSyuXfbmSBq2uVaNs1bUFn++dQsaVDEKqls+ziJkgeAMxkhdKtc//9w2JpxIBcNgdzB47l8CyAegNzg1NBh89odVCcrwuINif0Gp5V6kUBG8hRvJCiWW32Tl18DQBZQMIqx6a6zGXzl92+bvDAa0fasGpQ3FsXbSderfXueHOV0HwdiLJCyWS3Wbng16fcHx3LHa7nSEfD+CuPu1cjjFnmClfqRznT1zQHnv0jZ74l/Fn4HuPM/C9x4s7bEEocUSSF0qU1ORULideIS05leN7YrVdrHM/+D1Hkp/z4e/E7DoOgMHHwLMznqRNt9bFHrMglGQiyQslxv71B5k4cDqSBBVqhmG32rTnAssF5jj+7JF4bUUNOLickFJMkQpC6SFuvAolxuxx8zBnmDGlm0k4lUhUr7YElgugct1KvPjNMzmOv/epThj9jPgG+mLwNeba81UQ/uvESF4oMQKC/ZEkCYfDgcMBUT0jeXry4DyPj+jUlPdXvE2cegalTQPKVSxbfMEKQilR4CSvKEp7YCpgBE4Ag1RVTS6swIT/nqcmD2ZC3ykknUmmQ7/2NIoKz/c11cKrUi28ajFEJwil062M5L8HHlZV9aCiKB8BrwFvFk5Ywn9RxVoVmLRpvKfDEASvcitJvqGqqhZFUQxAVWBvIcUkCIIgFBLp+hZoN0NRlNuAvwALcIeqqqfdeFktnNM7giAIws2rDcS6e3C+I3lFUR4Fplz38GFVVTupqroPqKgoyjBgLtDW3RMnJaVit+f9ARMWVoaEhCvuvl2JIGIuHqUxZiidcYuYi4c7McuyREhIzqXE+ck3yauqOh+Yn/0xRVF8FUXprqrqwqyHfgYm3fTZBUEQhCJV0HXyFuAzRVFaZP29N/Bv4YQkCIIgFJYCJXlVVW3AY8DXiqLsBnoBQwszMEEQBOHWFXh1jaqq/wIt8j1QEARB8BhR1kAQBMGLiSQvCILgxUSSFwRB8GIiyQuCIHgxkeSFYmez2vhq+Hc822Q4U5/8DHOG2dMhCYLXEkleKHb//LKBzQu3cTkhhejVe1g0Y5mnQxIEryWSvFDsks9fwmp2dnSymKwknbno4YgEwXuJJC8Uuzt7R+FXxg+/Mn74Bvhw39OdPB2SIHgt0RlKKHYVaoYxZctHnDoYR9X6lQmuEOzpkATBa4kkL3hEYLlAtzo/CYJwa8R0jSAIghcTSV4QBMGLiSQvCILgxUSSFwRB8GIiyQuCIHgxP1ewUgAABEdJREFUT6yu0YGzX2F+3DmmpBExF4/SGDOUzrhFzMUjv5izPa+7mfeVHI68m2kXkXbAhuI+qSAIgpdoz020W/VEkvcBWgHxgK24Ty4IglBK6YDKwHbA5O6LPJHkBUEQhGIibrwKgiB4MZHkBUEQvJhI8oIgCF5MJHlBEAQvJpK8IAiCFxNJXhAEwYuJJC8IguDFSmzTEEVR2gNTASNwAhikqmqyZ6O6MUVRooApOGNOAoaoqnrSs1G5R1GU9wCbqqpjPR3LjSiK0hd4CzAAU1VV/czDIblFUZQgYBPwoKqqsR4OJ1+KoowBemf9damqqq97Mh53KIryLtALcAAzVVWd7OGQ3KYoykQgVFXVwYX93iV5JP89MEBV1duAg8BrHo7HHbOBoaqqNs/683QPx5MvRVGCFUWZCbzq6VjyoyhKVeADnKUxmgNPK4rSyLNR5U9RlEic29AbeDoWdyiK0gnoAkTgvM4tFEXp4dmobkxRlLuAe4CmQEvgBUVRFM9G5R5FUToCg4rq/Utykm+oqupBRVEMQFWgpI/ifYC3VFXdm/XQXqCGB0NyVzfgKDDJ04G4oROwVlXVi6qqpgG/4Ry5lXRPAc8DZz0diJvigVdVVTWrqmoBDlHCf5dVVV0H3K2qqhWogHOWIs2zUeVPUZTyOAcuHxbVOUrsdI2qqhZFUW4D/gIswJseDumGVFU1AT8DKIoiA2OBhZ6MyR2qqv4IoCjKWA+H4o4qOBPQVfFAaw/F4jZVVYcClJKBJaqqHrj6Z0VR6uOctonyXETuycoZ44ARwHzgjIdDcsdXwGigelGdwONJXlGUR3HOY2d3WFXVTqqq7gMqKooyDJgLtC32AHNxo5gVRTECs3Be2yL7dL5ZN4rZE/EUkIxzvvUqCbB7KBavpyhKY2Ap8Jqqqkc9HY87VFUdoyjKx8BinN+gvvZwSHlSFGUocFpV1TWKogwuqvN4PMmrqjof56euRlEUX0VRuquqenUk/DMlaDoht5gBFEUJBBbhvOnaLeurbomQV8ylTBzOMqtXVaL0TIGUKlmLCH4HhquqOsfT8eRHUZRwwFdV1d2qqqYrirIA5/x8SfYYUFlRlN1AeSBQUZQpqqq+XJgn8XiSz4MF+ExRlNOqqu7E+XXR7frJHvQzEAM8o6qqGGEWvr+AsYqihOGcb30EeNqzIXkfRVGq45xqfExV1bWejsdNdYBxiqK0w/ltrxvwnWdDujFVVTtf/XPWSL5DYSd4KKE3XlVVteH8lPs661OuFzDUs1HdmKIoETh/saKAXYqi7FYUZZmHw/IqqqqewTl/+TewG/hFVdVtno3KK40AfIHJWb/HuxVFecbTQd2IqqrLcE4tRQM7gU2l4RtIcRD15AVBELxYiRzJC4IgCIVDJHlBEAQvJpK8IAiCFxNJXhAEwYuJJC8IguDFRJIXBEHwYiLJC4IgeDGR5AVBELzY/wE+m2btOzxaQgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(X[:,0],X[:,1],c=Y,s=10, cmap='viridis')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.naive_bayes import BernoulliNB, MultinomialNB\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import MinMaxScaler;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "scalar = MinMaxScaler()\n",
    "X = scalar.fit_transform(X)\n",
    "trainX,testX,trainY,testY = train_test_split(X,Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "bnb = BernoulliNB(binarize=0.5)\n",
    "mnb = MultinomialNB()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MultinomialNB(alpha=1.0, class_prior=None, fit_prior=True)"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bnb.fit(trainX, trainY)\n",
    "mnb.fit(trainX, trainY)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.928"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bnb.score(testX,testY)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.704"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mnb.score(testX,testY)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "h = .02\n",
    "x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5\n",
    "y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5\n",
    "xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "Z = bnb.predict(np.c_[xx.flatten(), yy.flatten()])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "Z = Z.reshape(xx.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x24b12f1a828>"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEBCAYAAABi/DI2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xd4VGXawOHfmZoGSSCUQOjlpQoIIqKoKIpib4iwsuqCZXVdVKyogG0VcRVX0W9FRRdZ68JaUFcBEQSkSJHiq5TQEgIkgZAy/Xx/zGSYJBMSUkiQ574uL2dOmXnmMDnPvN0wTRMhhBCiNEtdByCEEKJ+kgQhhBAiKkkQQgghopIEIYQQIipJEEIIIaKSBCGEECIqSRBCCCGikgQhhBAiKkkQQgghopIEIYQQIipJEEIIIaKy1XUAx8gJnAZkAv46jkUIIU4UViAVWAm4K3vSiZYgTgMW13UQQghxghoELKnswSdagsgEWDxpJK6crLqORQghTggxjZoxaNJsCN1DK+tESxB+AFdOFkUHMuo6FiGEONEcU9W8NFILIYSIShKEEEKIqCRBCCGEiEoShBBCiKgkQQghhIhKEoQQQoioJEEIIYSIShKEEEKIqCRBCCGEiEoShBBCiKgkQQghhIhKEoQQQoioJEEIIYSIShKEEEKIqCRBCCGEiEoShBBCiKgkQQghhIhKEoQQQoioqr3kqFKqIbAUuFRrnV5q3xXAZMAAtgM3a61zlVJ/BJ4FiheW/kJrPaG6sQghhKg51UoQSqnTgTeAzlH2NQReA07TWu9RSj0BTAL+CvQD7tVa/7s67y+EEKL2VLeKaSxwJ5ARZZ8duFNrvSf0fD3QOvT4NOCPSqmflVKzlFLJ1YxDCCFEDatWgtBaj9FaLy5nX7bWeg6AUioWeAiYG9qdCTwJnALsAl6pThxCCCFqXrXbICqilEoE5gDrtNbvAGitr4rYPwXYWttxCCGEODa12otJKZUKLCZYvTQmtC1RKXVPxGEG4KvNOIQQQhy7WksQSikr8BnwodZ6nNbaDO3KBx4INXAD3EWwhCGEEKIeqfEqJqXUPOBxoBVwKmBTSl0b2r1Kaz1GKTUceC3UNvErMLqm4xBCCFE9NZIgtNZtIx4PCz1cRTkllFDD9qk18d5CCCFqh4ykFkIIEZUkCCGEEFFJghBCCBGVJAghhBBRSYIQQggRlSQIIYQQUUmCEEIIEZUkCCGEEFFJghBCCBGVJAghhBBRSYIQQggRlSQIIYQQUUmCEEIIEZUkCCGEEFFJghBCCBGVJAghhBBRSYIQQggRlSQIIYQQUUmCEEIIEVW116RWSjUElgKXaq3TS+3rDcwAGgLfA7drrX1KqdbALKApoIFRWuv86sYihBCi5lSrBKGUOh1YAnQu55BZwF1a686AAYwNbZ8OTNdadwFWAY9VJw5xYmvU6RQ6XHg9SW271HUoQogI1a1iGgvcCWSU3qGUagPEaq2XhzbNBK5TStmBs4GPI7dXMw5xgmraoz/9bptIp2Gj6H/X0yS371bXIQkhQqqVILTWY7TWi8vZ3QLIjHieCaQBKUCe1tpXars4CTXrNRCrIwbDYsVid5LStV9dhySECKnNRmoLYEY8N4BAlO2EtouTUM7WjfjcRQD4PW4O7dB1HJEQoli1G6mPYjeQGvG8OcGqqH1AolLKqrX2h44pU0UlTg57ln+DxWKlsepD1s/L2Lfhx7oOSQgRUmslCK31DsCllDoztOlG4EuttRdYDFwf2j4a+LK24hD1366lX7H27b+Rueq7ug5FVMGqTC8z17n4aa+v4oPFCaXGE4RSap5SqrgieRTwolLqFyABeDm0/c/ArUqpTcAg4NGajkMIUftWZHh5YXkRn/7mYcqyQn7K9NZ1SKIG1UgVk9a6bcTjYRGP1wH9oxy/Azi3Jt5bCFF31uz14fYHH3v8sDbLx6mp9roNStQYGUkthKiyU5racFqDjx1W6NG0Nps1xfEm/5qiXnEkJNF60DBsMfFkrJxP3u5tdR2SOIoz0uyYwLosH32a2+jfQkoPvyeSIESdiW+aRsO09uRu34wrdz/Neg2k900PYViCBdtWA4ey5G93UpSTVceRiqMZmGZnYJokht8jSRCiTjTq2IN+t0/GDASHwCydeg9drxqLxWotcVxi606SIISoI9IGIepEqzOHYXXEYIuJw+pw0rz3mfhchSUPMixYHE5iklLqJkghTnKSIESdyN+7A7/HBYAZCGAG/Kx793kKc/YR8Hk5tGsLZiBAz5F/5dzJM1FX/KmOIxbi5CMJQtSJbd9+wuG9uzADAQyrjQ5DRxCT1Jg1M57i63uvJGvdUmxOJxaLFcMwaHvuFSSktqnrsIU4qUgbhKgTpt+HxWoLN0hbrDb6/OkxzICPrPXLydZrSh5vBnA2SCI/c0ddhCvESUlKEKLOHPhldbiaybDasNrt2JyxtOh3Lm3OuZzsX3/GNE0Cfh/5e3eSu21jHUcsxMlFShCizuhP36EoZz8JzVvR6oyhGLZgV0nDMGjYsj171y1l0RNjcDZM5tDO3zD9MtePEMeTJAhRd8wAOxd/DkBsclOadD8NwzAAMCwWYpJTiG+SSt6edEkOQtQBSRCizsUkNSa5Q3cATDO4VEjA56VBi7b0vvlhMAyW//1e8vfuqsswhTjpSBuEqHMpXfpihHorGYZBXkY6u3+cj9XuwB4bj80ZQ9oZQ+s6TCFOOpIgRJ3LzzpSMvB73RzcvomYxEbhUdYBr4ei3P11FZ4QJy2pYhJ17uD2zWz8aDqtz7yYvN1biW3UlMademFYrJimiWma5O3cUtdhCnHSkRKEqBcyVsxn+Yvj2fTRayS3747V4QxXOdmcsfS55eG6DlHUEUdCoky3UkckQYh6J3frBgJ+X7jBGoI3iY4XjcRid9RhZOJ4a3XWMM6dPJOzH3uD7sP/XNfhnHQkQYh6xbBY2L95NRiWcJdX0zQxLBY6XjySC6Z8TEyy/Jo8WXS9cgxWuwOr3UHL0y8gJqlxXYd0UqlWG4RSaiTB9aTtwEta61cj9vUGZkYc3gTI1Vr3UEr9EXgWKJ7H+Qut9YTqxCJ+H3qMHEdq7zPDyQE4MjbCMMBioefIe1j5qnxdjgev32TWBjdbc/0MaWdncLsY7HEN8BTkQUQJr7KS2nWl1+jxWKx2Nn40nX0/Lz/q8X6vB6vDGX4e8Mma18dTlROEUqol8DTQF3ADS5VSC7XWmwC01muB3qFj44AVwO2h0/sB92qt/12N2MXvUNNu/bA6YgBKVDFFJoxGnU7hwqn/IXPNEjb8+6VwbydR82ZtcPP1Vg9xDRLY17I/Xe+5lVatUnEfPMCSZ+8sO0V7BfrdNhF7XAMAet/0APMfGYXfXVTu8WvffpbeNz+I1e5g839m4MnPq9bnEcemOiWIIcACrXUOgFLqY+Ba4Ikoxz4MLNJaLwk9Pw3opJR6BFgH/EVrnVuNWMTvRO72zaR0ORVLxLQbxYnCNM1wwzUOJy37n0fe7i3sWPRpXYZ8wkhIbcPSDJOpn23EasBf+8fSu9nRbwFbc/10Vh1Y8N/pJMTHAsF/k5jkJnS6ZDSbP3m98gEYBlZnbOQGrI6YoyaI7F/XMv/hGyr/HqJGVSdBtAAyI55nAv1LH6SUSgRuBXqWOnYqsBR4BngFGFWNWMQJLrl9d3rccDeGYZD502KanTIAe2w8UKqKqZSmPc/A5yrE2SCZjFULcB3MPq5x11f2uAa0GjgUv9fD/k2raNy5F2kXjuKyPtfg9gQT7pRlhbx3RYPwdbXFxNLy9AswA352L/+GgNfDkHZ2+t00ivi42DLXPyG1NQnNW1V+hLtpsu3bj2k3+CpM0yTr5+V4DsvvwvqsOgnCAkRWQhpAtLL+H4C5Wut9xRu01lcVP1ZKTQG2ViMOcYIzLBb63T4JW0wcAM6GyWSsWUyrARdGTQqRktt3pVHH7hAI0O68q1j0xJhjrvY4UVnsTmKTUyjKySLgOzJXlaNBEgPGTSUmKQXDYqHr1bcCcDi/kEBEtZ3HD0kdetJ20MUU7M+kea+BxKWkgmmSeuo5/DjtAc5t46BpTD4Bvw+LJViqKy7RJbXrwsDx09i55At+mftmpWL+7Yt/kbHqOyw2O4f3bKupSyFqSXUSxG5gUMTz5kBGlOOuJFhKAMIlilu01i+GNhmAzMR2ErPYHCW6r1rsDrwFhys8zzAMrKGqKCxWDKuNBi3bk7t1Q22FWm/EpTTnjHtfxGK34y3MZ+nUcXjy8zjtz0/SqGOP8NQlkRokxHHTDZfyrw/nEfB6GX12Bwbc/QyGUdyZ0Qw/Tm7XNThQMeAnZ+FsDnZoR1KbLvjcRRTuzyS5XRds9mDjcZtzruDXz98pkaSOpiBL5tQ6UVQnQXwLTFJKNQEKgGsIViWFKaUMgo3YyyI25wMPKKWWaq1/BO4C5lQjDnGC83tcZKxaSPPeZwGwd80Sdi35gvbnXQ2hNogSXV6jlCpM08TmjOX0u5/lYPovrHz10fBaEycai91Bu8FX4WiQzM7Fn1Owb3eZY9qdfy32uAQMiwWL1U7786+hRb/BOBoklbhWULJqbtrf7mXUeV3I37KevsOuxmKxhveZJpgBP6ZpUpSdhRnwA+BzFbLylSO9xs5+9J8Q8ZoBn4eA31+zF0HUC1VOEFrrPUqpCcBCwAHM0FqvUErNAx7XWq8i2LXVo7V2RZznV0oNB15TSsUCvwKjq/UpxAnv5/deYtcPXwJwMF0D8OMrD9Nj+J3ENWkRLCFUUN1kmiYWi4WGaR1odebFpC+sv787LDY7TXucjt/jYv+mVSX2DRj3PA1btgPDQsvTBrPoiT/hLcwnLqU5SW27kr93By1OO+/ITdowaN5nUInkAMEuoUWHsolv3Dx0WHCfKz6V6T9+T/us//HcxE5YLBZM08TnKiRr/TJMv4/f5s2KGrfVEUNs4+YlktC6d6ZUqcurqP+qNQ5Caz0bmF1q27CIx/sIVj2VPm8xcGp13lv8/hQnhmKe/DzimqaFb2AQvaG62JF9BhZrPZ5mzLAwYNzzxDdLw2p3EPD72bNiPhs/fJUWfc+hYVqHIw3HsfGc99R7bP3fB7Qfcg0mBKvVIgYSGhYL9viGJa6NaZpYbDbiGzfH73GHxxL4/QHuffQlNuntOJ12XC43zz/yJ4py9rLu3akUZe89auh+j4uCfbuJb9ICgMIDmezbsKIWLpKoD+rxX5E42bUacEHUHkylq5wAAl538EbojMWVu59dS788/gFXUlxKc+KbtcLmDI73sFqstOh3LpgmjTr3KjNI0LDZ6HjxyPDz0gzDCHcLLtEVGCNU9RYTvk4Wi8Etf7ic8Y9Nw+328u13K/jO/X24Oqkylr/0AG3OvgyAnYs/q8olECcISRCi3jqcuSP8uHTbQ+SNMODz8vPsaexduwRHQiLuw4fArJvBc0ntutLn5oewOmLYsWQecY2akp+1m23ffoTp92F1xtJh6PVY7fYS51lsdtIGDsUwLFHbWY5WcjJNEwIBsFjKXKNo52dnH8Rht2OzWxnU1okZKDimz+grymfr1zLG9WQgCULUW3tWfkfPkeOAIwPmjvw6PsKw2ohp1AQzEMCdd3z71Tdo0Y7YRk3J2bIen6uIvrdOxBEfHCnc4YLrgnEHArQbfCUHd/xKTFJj4lJSw1OZY5r4vG5sdieG5dimRitOAPs3rSaprcLwWrHYbKHXMSjI2sXhvTtI7TMo3Dsp4PfSYdunjOpqISXOYEDTbIIdCYUoSxKEqL8Cles2aRgG7c+/lj3Lv8HqjKuwHr26mvYcQEKzNEyg49AbMAN+fK4C9GfvYo9LKBufxYI9LoGULn3C8Rb/PxDwkz5/DmlnXEhMUuNye2iVV3ryuYvIWv8DP89+kUYdepCftQtX7j4sNnt4WoqsdUvpdu3t+L0e1r3zPI0cfi7r7CzzPkKUJglC1Gv7N62iSbd+FR5nj2vAeU+9R8DnZf+mVax565kKzykW36wVjTp05+COX0sM3rLFxuN3F2GPa0BStwEs/zWLFq1a02v0aCxWO4bFEv7Vb3XE0OsP95ZpG4kUtf3AYqXD0OGhX/gmplm2vaXkY5Os9T+S0vVUbKE1Mw7t+A3P4YPsXbsk4pWPTF+xd80S9q5ZwkFXgJdXFpGZH6BNooXOjWxc1NFOnE0mdRbRSYIQ9dqvn79DiuqFxeaIetMtUe1kGFgdTpr0OI3znp6Nz1XA2pnPkber7Gp0DdM60PL087HaY0gbMARCdf9r3vob+zcsZ+D4l2jQoh1+j4uAz4fP6qBLwCTvcD620HxCZiBw5P1L1f8DJZJFeW0IwRlqreUmlmApwUVR7n4CXjf605lk/7qONmdfSsO0DuxZMZ/DGdujvvaOQ35eWVWEywd/6hXDp7+5WZflxwSyCvysyPAze6ObKefF0z7ZGvU1xMlNEoSo1+yx8QR8Piw2R4l2iGLRbrwWqx1nAwfOBon0Hfs4Cx8vOcwmvlkrzrjv7xgRg8SKk0zXK/9ESpc+NGjZPriaXUwcAX8AhzVUh28GKCx0ERcXA1HaQyD6TT7aMaU/j2maHNyhiU1uEuy2arFiCZVQ1r87hbzdR0o3lZmg8KklhWQXBWN5dlkhCfaSc+MEPw/MWOvimcHxFb6eOPlIghD1Wu62zRzOSKdBy3bBG6bVhknZm27kDTeSPb5sm0CTbv2iTkUB4ExsXGIOqOBiRUce787Yz3eLVzH2xsuwRaxTUDpplY6jdCLwFBzGYrUGl1a12jADfgoP7GXFyw8S8PmIS0nl1DGP4kxszLZvPy6RHKIxTRO3z2RnXoAYO2zNDXDQVTKGXk2tLNxZtjurNFGL8kiCEPWaGfCzbcEcet7wF6yOYHKwlJMcipUYTewP0PL0IRzOSKfVGUPBgP2bfir7PqHXsNhs5d7sDcMgObEBVww9k4yVC2g18KJyE1XkczMQoGDfbuIaN8Ow2tk2/2N++/xdILi2RcehI3AdPMDm/7wRns+o8EAmS569s1LXaH2Wj2eXFeIqp03fADokWxnbJ5YN+/PJdYMv1As4wQ5j+8RU6n3EyUcShKjXrA4nvUePL7GqGJS8EZe+iUeyOWPodu0dWO2O8NQUaQMuZPeyr2l5+hAAXAeziW3UNOr5pd+zWZNk1r/3EhmrFtC0x+k4GyaXKBl4Cw/jyc/DFhuPMyGRQ7u2svad5yg6kAmGBYvVUmJSu5zf1rPit/XVuUS88GP5yQGC1UrdUwxi7BZeHtqAzdl+UmINUuIsOKxlE64QxSRBiHrNUmp8gGmamH5fsFqmVJKIFJkwikcSh4+xWPF7PXx9zxUADHn2gzLvWzLhmLgP55G1fim/fv4u3oJg91H937foccPdWO3BBvTC7L18/+TY8LxExbOhHnnRAAFfzQ3g85smuUUm7koMgv5qm49rupo4bUaFiwQJUUz6t4l6zVuQx56VC/B7XPg9bnYs/jzcOFzer33TNIM9j1wFmBEjqsPtAmYAd14OADHJTbHFxpcoBZimScDrwXM4F7/Hja+okJWvTmDjB6+EkwNA5k+LyFq/FL/HzaGdv7H87/eVmLTuWKavOFb5HpO/fp3PXV/ll215jsIXgO25MuOqODbyU0LUexv+/TLpC+cQ8PuJS0mlzaBLj3q8YRhY7fZwe0Ig4GfHos9I6dKH2OSmHPhlNekL5wLgPpxL8A4bSjamyar/m0Tert/wFuYTk9wUd14uAa+7zPuYgQDr3nm+hj9t5SxI97Cv0MQXCEbeLcUgq8Akzm6hRYLBjxklk4FpQpN4+T0ojo0kCHFCKF7Wss8tj1Q47XexI9VPFvweF0v+9ucyx5g+Lz/NeJqeI/+KaZqs/9cLHNi8Ory/tkdlV5XdYoSL/xYDWjW08dS5wfEZt88rudiSBXhwYByNYyVBiGMjCUKcUEx/ydbYyLaC6D2I/AR8Pvb9vLzc19z383LmP1z+/vrovHZ2lu7xsmm/nxYNLFzf7UgjfquGFvYVHilBtEmySLuDqBL51ogTyob3X2bAPS8EeyWFRPZiCvh94QFw+Vm72LXkS3K2rOdwRnpdhFtrnFaDJ8+JJ2CaZXohjegew7p9BfgDYLXAyO4y75KoGkkQ4oSSt3sbP057gP53/Q2rM6ZsL6bQADi/10Pm6u/Z+cO8MqWOE5lpmmw64Mfth1OaWsnMD/BqaDqNW3rFcEozGx2SrUw5L54N+310amRFNZY/c1E1RrT5beqxtsD2b+4+n6IDGXUdi6gGb8Dk+WVF/LTXR6uGFiYOiqOh08A0wWqpuI3BkZBIh4tuoNUZQ7HY7GW7uQb8FGZnEde4Oe7DB1nxyiMUZO2qrY9z3MxYU8T8dG+wcdoITpURCP0JO63w5qUNiLPLuAZRUmxKCy54eT5AOyC9sudJq5WoE4t2ePl5n4+ACbvzgrOMjpp7mOvnHOb9ja4Kz/fkH2Lzx6+z7O/3sX/jynAVk2maBPx+An4/MYmNMSwWnA2TGXjf32l3/rW1/bFqjctnsuOQn6+3eXH7wW8Gu64GIn7fBUzIc59QP/hEPVetsqdSaiTwKGAHXtJav1pq/0TgFqB4FZc3tNavKqV6AzOAhsD3wO1a699PPYCokNt/5OYWMGHDfn94+oc52sN5bR00rUS3zMN7trH6jSfoNXo8qaeejSf/MLaY2OBcSxED7GwxcXS8aASF+zPIWr+0Nj5SrdlXEOD++QV4A2aJhFBakziDZvFSehA1p8olCKVUS+Bp4CygN3CrUqpbqcP6ASO01r1D/xUnkFnAXVrrzgS7cY+tahzixHRuazspcQZOK8TZwREx27RhBH8hHwt7bAJ+jxt7bHxoUj9rmZ5NFpud+KYta/BTHB9fbPFw2GPi8gVHbMRF+Vlnt8Ddp8VWuguwEJVRnRLEEGCB1joHQCn1MXAt8ETEMf2AR5RSbQiWFMYDzYBYrXVxv8KZwGTgtWrEIk4w8Q6Dly9MYH+hSXKswepML9NWuDBNOL+tndSEyv92SWiWRnKHHticwUnniqubfG4XWeuW0rz3QAKBAAawd90PtfFxasW+ggBTlxey81DJ6TlKT61hMeD0llY2HfDTsoGVeIckCVEzqpMgWgCZEc8zgf7FT5RSCcAa4H5gC8FE8BjweZTz0qoRhzhBWS0GzROCN7OBaQ76NrfjDUDCMd7gvIUFZdZz9uTnsW3+R2xfMJdt8z+mYct25GzdiOvggRqLv7a9vLKIrbmBMjNplC5dmSYs2+1n+R4/C9K9TLswXkoSokZUJ0EE10g8wgDCP3W01vnAsOLnSqkXgLeAeUc7rzKKbptFgUfmlfk9MoCCYzynANhS5KOzLUBwYTmDQkdDNpx6B5x6BwWAERNgUGMfFgN+Pmjht/z62fWzMD+fPdu3s27pUtJdH2KSX+E5JqGkYcJel5Ws2z6hQWJirccqTiCOqq0YWJ2/kt3AoIjnzYFw31OlVGtgiNb6rdAmA/CGzkst7zxxcsnLzWV/RgZpHTrgjCm5LkFhfj7uoiKSmzSp8HU2HLKS4jRJtAd/e6w/WPKr3b+xj+Kll3skBdhZaOIO1K9f2Rnp6UweMwZXYSEB/5EfQHanE6+77FxQMXFx9Bs8mM2rV5O7fz8ADRs1Ir5Bg+MWs/h9q06C+BaYpJRqQvBH3DXArRH7i4ApSqmFBPvd3gnM0VrvUEq5lFJnaq1/AG4EvqxGHOIEVJifz3N33822jRux2mwkNW7Mk//6V/iX74/z5/N/kyZhAv0HD+b2yZOPvlYDBgv32UiwgTsA3lI3/9ItGhYDcvbt492pUzl88CDX3XEHXfr0qeFPeWzmvfceRfn5ZVaji5YcrFYrPq+XVQsX0rhZM8697DKwWLjippvCy5QKUV1V/iZprfcAE4CFwFpgttZ6hVJqnlKqn9Z6P3Ab8BmgCZYgXgidPgp4USn1C5AAvFyNzyDqCa/Hw4G9e0v8+i3Ph9Ons33TJgD8Ph+HsrNZMX9+eP+7U6fi9XjweTysWrSIXVu2ABDw+1k4dy73X3cdYwcP5uWHH8bn9YbOMsj3GSWSQ7I9wMAULwe9wWoYvwnbDlso8hv8/b77WLNkCb+uW8fUceM4fPBgzV2MY7RryxYO7N1bJjmUZrFYSGzcmAbJyfi8XlyFhWRnZdF/yBBufvBBGjVrdpwiFieDalXEaq1nA7NLbRsW8fgT4JMo560jokFbnPj27tzJ5LFj8RQV0bh5cya++WaZqo61S5bwvw8/pGX79mVuhgHT5NO332btDz8wZsIEHI4jcy2ZponDGZxPaMYzz/DDl1+Gk9BPixezcO5cLrjuujIx2Q2Ts5v6sFuCYy125xSyOj8evyX4tc/avftIMjMMDuzdy/ply8g9cIABF1xASvPmNXqNyrNt82aevv32EivNlScQCHAoOzu4HobFghkIEPD7adio0XGIVJxs6mdLnTjhzHnzTQoOHcI0TQ5kZrLkiy8YOmJEeP+urVv5x4QJeFwuflmzhvbdumGxWjEDgfAiPTn79nEoO5vXJ07kjiee4O/jx+MqKODS0aNp3ro1AMu/+aZECcXv9ZK5Y0fUmGJtwQQ0//uV3HLXk+zPPkhMXBznXnklHpeLngMGsH7ZMgAaN2vG9599xuLPP8fn8/H5u+8y9eOPsdntZO3eTbNWrYiJja2Va/fTokV4XBWPHo9kmiYt27WjIC+Py2++mVYdOtRKbOLkJglC1AiHM7g0qOn3B5f5jCgBAOzeujVcYvB6PPy6bh2maWJ3ODjljDPYuGoVroIC/H4/GenpdO7Vi9e/+YZdW7ey6NNP+erf/2bIddcRrRViZ6j6qbR8r8Gv2zO4ZvSDuD3BaqiiggK+nD0bTBOr1cqIv/yFho0aceqgQTwwfDju0I064PezZskSZk+bhs/rxe5w8MTMmaSkpkZ9r+po1bEjVpsNfyVKEJEGXHABV9x8M8u/+YZP/vlPThs8mNadOtV4fOLkJa1ZokZcc9ttpLZujWEYdDrlFM6+tOSqbw2Skko0thZXLnk9HjLS04mNiwvvO5iTw8aVKzl44ABPjBnD1++/z0evvcbbzz5L1759y7y3AXjcbg5lZ/PkrbcycC2TAAAf20lEQVQy/tpr+fHbbwlg8PFPGeHpv4+8efDd/X4/X86ezcChQ4mJi6Njjx7hxBYIBNBr11KQl4ersJCCvDzmf1KmtrTa/vPGG7w2cWKJ5JCUmECHdhUPDYpv0IDP332XGU8/zdy33uKJsWPJ2nXiT0go6g8pQYhqy9y5k9WLFnHdHXfQZ9CgEr1o3C4XZiBA7v79JX8lh27SDqeTngMG0DApif+88QaBQICAz8fXH3zAkGuvDZc6PG4365ct42+zZ/PmM8/w8/LleL1erDYbv65bx9jBg0tUPb0yYQIJSUnMfmU6rojqG4vFQiBwZNhNzr59BPx+LFYrt06cyJwZMziQmcnFN9zAzytWYLXb8Xk8WG024mt4bMGm1auZ++abZRqmD+cXEl9BdVZahw6ce8UVPHnrrSWqp/S6dTRr1apG4xQnL0kQolr2Z2Tw+B//iMftxm63c/VttzFs5EgAFsydy7vPB9dsvvD668tWoRgG195+OwMuvJC3//a34CRMgM1mo0WbNjRLS8NdVBQ+3ON2k5CYyF+few7TNNm7axcPjRhR4oYfafpjj5GXG5wn0ma3M+Saa3jmkbFcOPgasnMOYbNaSWnSiH888gijxo0jJTWVG/7yl/D5aR078uu6dfy2fj2qd28ujNIQXh0r5s+P2mvJ7w+wZ+/+cs/r0L07k94KDi/qftpp7Nm+HY/LhWmatOvSpUZjFCc3SRCiWvTateEptt1+P998+CFfv/8+cQkJZKSnh3/V/++DD2irFOlah8+1Wq1cPHIkD91wA3t37Agf6w8EOJyXRyAQwO5w4PV4gGD7wbZNm2jfrRuGYdAwOfmoYyPycnLCjw2LheatW+PKzmLtolnM+Nd/MQyDbel7ePfDL9m5ZQsvlKpCiomN5cGXa74Htt/nY8GcOaz54djnhTIsFtwuF7u3bSOtfXuuve02EhIT2bVlC2dfeimtOnas8XjFyeuEXDDo042ZMtVGPbHj1195YswYPG43NoeDgM8X9Re9zW7nhf/8h/HXXhtui4iJj+fZ2bMZd+WV4SqnYhabjRkLFvDwyJHsz8gITrZnsWB3OBg2ahSHsrM5fPAgq7//HrPU+1ms1jJjMZKbNaNzz57YC7KZ8+5zBAIB8g4XcPoFN5OTm4dhsfDO0qXhhBMIBGplwJnX4+Gx0aPZs317pc+xWCy0bN+e/RkZuEIlqviEBP4xbx72Up0BhIgm3mHl8u6pcIwLBkkJQlRL606duH3SJBbPm0dykyYsmTcvXCdudziCbQqBAD0HDMBitXLWsGEsnDsXTBOPy8X3n39OW6XY9dtv+CNu6qbfz7QHH8Tn8xEoXgwoEMDjcjH3zTePGlPztDQySnV9bdy0Kds2bWJ/Rgbdz7ieLqoD6ZnZFBR5cMbG0vP004Mlis2bmTpuHAV5eVxw3XWMHDeOhXPmsHrRIrr378+wUaOqNRHexhUryNqzp3IHGwYde/SgTefOXDh8OA+NGBFOpB6Ph8O5uTIwTtQqSRCiyg7l5PDk2LEcyMykWatWjJkwgcz0dLZs2IBpmlw5Zgy7t25l9aJFrF+2jEdvvJEzLrwQm82Gz+vFarUSExcXnIPJMMLrN0Cwn//G1avxhaqXjkXp5ACQtWsXZ11yCd9+/DH7sw9ycPUGnpk9m1/XrsXucHDa4MEA/HPy5PCI6u/++18aNm7Mf996C4/LhV63jpi4OM6/+uoqX7O4hg0r1Z11xF/+Qk5WFt99+inbNm3ih6++omvfvmzdsAETaJaWRlIl5qgSojqkm6uosnmzZnFg7178fj9Zu3fzzYcfhksBhsVC1q5drF2yBI/Lhd/noyAvj259+9K+WzfsDgdd+/alc+/epGuN3+cr02BrqcEpqwvz84mJjcXhcASnBjdNvpo9m0GXXMKACy7Aagv+VvKGp+0ADIM927aFS0Qel4utGzdWK442nTtHHctRWt+zz2bjypV4XK5gdZlpcs4VV3DTgw/yh3vu4fEZM2TOJVHr5Bt2EvtlzRoeGD6cB4YPR69de8znl16DwVVUxPZffsHr8eB1u1nyxRe0USq83+f18skbb/DYP//JW4sXc/9LL5GckhJ97ibD4Kxhw0pU5xiheYiqwu/zsezrrykqKiLg9+Nxu/n+s89YuWBBicT0x/vvx+F0YrPb6dG/P0Ouvjr83OF0csYFF5R57TVLlvD65MksmDu33LmUCvLy2LdnD7+sWRPurVWepmlpNG/dms69e2MPTTESCARopxRnDRvG4CuvrLVR3UJEkkbqk5Tf5+P2Cy7AVVgIBKeO/r9vv8VirXje+PxDh7BarXi9Xp689Vb27dlDauvWPDBtGg9cf32wa6ph0KhJE+6fNo0Jo0Ydabg2DN5dtgzDMNi4ciXvPP88ebm5FOTllX0jwyA2Lg6Px4PFMLh45EgO5eSwZN68Yx51jGEEx0CUSkbOmBiu+NOfGDp8OKu//57P332Xxs2bM/yOO2jZvj2GYZCuNb/89BMde/SgY8+eJc7fvHo1U++5B4/bjSMmhuvvuosLr7uOnKws3n3hBfIPHaLPWWfxyT//iWmaNE5NZd+uXeUmEqvNxuS336ZN5854PR4+ffttMnbs4Pyrr6Zbv37H9pmFCJFGanFMvB4PnoiRzR6XC6/Xi7OCBPHx66/zxaxZAPzh3nuZ8uGHuIuKcMYG10N+YNo0Zr34Ig6nk1sefpimaWkkpaRwMDsbi8VC606dMAwDt8vFi+PHh6e2iMo0KSooIK1DB+55/nn0mjXMmx2cGzKuQQNiYmPJy82NmM2V8AR2pRkQtaTidrmY88YbfDR9evimvXvbNgJ+P+NffBGAtkrRNqIkFGnLhg34QsnK43KxaeVKLrzuOl647z52b90aHpFdLGvnznAVFwR7XFmtVgKBAAmJiVx3++206dwZCDbyX3PbbWXiffnBB9Hr1qF69+avzz6Lo9Q6GkLUFEkQJ6mYuDjOuPBCVn33HQD9zz+/zII9pR0+dIgvZs0K35D/9cILweqOiGkyOvfqxRMzZ5Y4b/Lbb/PV++/jcDq56IYbgOCYBn8lpgUH2LNtG1abjQ+nTw83Wvt9Pv5wzz0s/uIL1i9fjt/nwxkTQ4eePdm0cmWZ17BYLOW+n9frLdHNNuD3k/7LL/z7H//gUHY2w0aNonWnThQcPky61qS2ahXuPdStXz9sb76Jx+/HERND33POAWDvrl3lDuCLTGB9zzmHvzzzTKV7Rn01ezabf/oJr8fD5tWr+fqDD7jsj3+s1LlCHCtJECex2yZO5PxrrgGgY48eFR5fulHUarVW6saWlJLCiLvuKrEtsVEjeg4YwKZVq8A0OWvYMLr27UtBXh5tu3Zl4k03hY81TRPDMIhPTORgdjYQTBCvPvZYuKrJ7nAw7MYbSWneHL1mTYkqKGdcHBaLBVdhYdTSRekxGACHcnP53wcf4PN6Wf399wy9/no+nTkTg+CYjgf/8Q869+pFh+7defCVV1j3ww+kpKaS2KgRBXl5nHnxxSz96is8bnf09wzZvHr1MXWbzc/LCydon89H/qFDlT5XiGMlCeIkZhgGnUrVqR9NfIMGjPzrX3nvpZewWq3cNmlSlccEGIbBuClT0GvXYrPb6dijx5FBan5/iWU2bXY7sfHx/OHee5n53HMcPniQooKCEjder8fDqoULuWjECGx2O36fD4vVSs/TT2fYyJFMnziRovzg+s7FVTxHbX8zzfCN2AwE+Oydd4JTkxOc8uPzWbO4t1cvADqfcgpej4e/33cfVqsVm93O5JkzSW3dmv999BEHMspfUTfaanFHc8Hw4SyZNw+f14vN4WDItdce0/lCHAtppBbHLBAIBBesqcFuqKVt/uknZjz1FKZpcsXNN/PetGl43W6apaXRvmtXFs+bV+YcwzBwxMTQZ9Ag1i5ZQmqbNtw7dSobVqzgjaeeCrdBGBYLhmGEn9udTgzDiLomg8VqJSY2Fo/bXaKt44Lhwxl9333h51Puvpuff/wRCCa0hMREXIWFuIuKyk1EFouF0ffff8zjKlyFhcE1KtLSSlTvCVEeaaQWx01t97/fuGoVM597DrvDwZhHH+X9f/wj/Ot/z/btZKSnRz3PNE3cLheNmjThjYULw9sPHjhQ+sAS7QOJjRqRs29f+LnNbsditdKhe3d69O/Paeedx9/+/Gdy9wcn0Itv2JBrSzUeN2nZEpvdjs/rxQQOZWcftYTicDoZPX4851x+eWUuSQkxcXHhhmwhalO1/tKVUiOVUpuUUr8ppe6Msv8KpdRapdQ6pdRcpVRyaPsflVKZoX1rlVJPVycO8fvhcbt58b772LtzJ3u2b+f5ceM4sHdviWNK33gbNW9+ZIEi0+Sr99/ng1deCe8/Y+hQ4hISiI2Pxx4a01DM7nTSo3//cDuE1Waj18CBNGralMydO0lITCS1desSYxe8Hg/7S1UbjbjzTlTv3ljt9gqrrxxOJ8P//GfOvuyyY7s4QhxnVU4QSqmWwNPAWUBv4FalVLeI/Q2B14BLtNa9gPXApNDufsC9Wuveof8mVDUO8fviLioq0duoqKCAXmeeedRz8rKz6X/++eHnAb+fr95/n0Oh2VwbN2vG1I8/ZtyUKUz96CPadukSnuJj6PXXc8Utt5DYuDE2u53mrVuTvXcvWbt2cXD/ft578UUy0tPLlJpKP49NSCB3/378Pl+FYzRufvBBho4YUatVdELUhOpUMQ0BFmitcwCUUh8D1wJPhPbbgTu11sUzk60HRoUenwZ0Uko9AqwD/qK1zq1GLOJ3okFSEqcOGsS6ZcvANDnz4osZeffdHM7JCfb979WL3du2lViH2u/zcdGIEaz+7rsS60dEDvqLb9gwPNDskddeY/OqVcTGx4cHvr303/9ScPhwcL2Jyy8PlwAsViuHcnIY+9hjvDh+PF6Ph37nnsvB7GwaNWuGu6iIpMaNsVitwSqoCtr0Op1yCqeGusIKUd9VJ0G0ADIjnmcC/YufaK2zgTkASqlY4CHgHxHHTgWWAs8Ar3AkeYiT3J1PP82Wn38OtwMYhsHdzz4b3v/LmjU8e+ed4ZKGMzaWVh07cvvkybz2+OP4fT6uv+suGpSzApzNZqPngAEltlmsVhokJQFw5S238N6LL2KxWmmWlkannj2x2e38c/58ln3zDW898wyrFy3C63ZjsdlITkmhW9++FIVGpdscjmCDdpRksX3zZsZdfjkTXn9d2hFEvVedBGHhyNLCEBqsWvogpVQiwUSxTmv9DoDW+qqI/VOArdWIQ/zOWCwWOoe6kEbTpU8fht14I//74AMaJidz/0svYbPZ6HfOOcz47jtM06xWQ/p5V11Flz59OJSTE04OEEwiX/zrXyVGoAc8HrKzsljy5ZfhhNAkNZUDmZnhhY4wDAiN5fB5vfi8Xua8+SbjnnuuyjEKcTxUp5F6N5Aa8bw5UKLlTimVCiwmWL00JrQtUSl1T8RhBnCME+uIk93wO+5gxnff8fc5c0ht0ya83QjNuVRdLdq2peupp5Zo0AZIad4ca5TpSCLbEwKBwJHkENp39dixOEIT71ltNhIaNqx2jELUtur8JX0LnK+UaqKUigOuAb4q3qmUsgKfAR9qrcdprYtLG/nAA0qp00PP7yJUFSVEffenCRPocuqpJDVpEm7jCPj9NGnZErvTiSMmJry2RDFHTAyX33QTfQYNwuF00rpTJ4b/+c91Eb4Qx6TKVUxa6z1KqQnAQsABzNBar1BKzQMeB1oBpwI2pVTxcM9VWusxSqnhwGuhtolfgdHV+hRCHCeJjRrx0CuvsH75cv7x8MPh2XD9Ph/Pvf8+CYmJzH3rrRLntO/aFavNxp1PPVUXIQtRZdUaKKe1ng3MLrVtWOjhKsopoWitFxNMHkLUWysXLuSbjz6iVYcOXH/nnVhsNv45eTLrly8nrUOH8GA7w2IhsVEjnLGxxMbH0yApKdhQ7fFgtdlQvXvX8ScRompkJLUQUaRrzeuTJuFxudjy8894PB5ad+zI6kWL8LjdbN24EdWrF7u2bCE2IYF0rfnrZZfR79xzGTNhAptXrWLzmjV07NGDS/7wh7r+OEJUiSQIIaLYs21buOHZ6/GwffNmkho1OjKTqtdLkxYteOiVV7j3qqvCk+6tWbKE3du2cf+0aXUWuxA1RZYcFSKKbv36YbVasTudOGNiOOeyyzjn8suJa9CA2Ph4YuPiuGjECIDwetYQnAYk8rkQJzL5JgsRRXKTJjz93nus/eEHUlu3JqlxY9K15oGXX8ZVWEirDh1ICA3Eu/7OO5n24INAcFU5vWZNuSvQCXEikQQhRDlSmjdnyDXX8NPixTx+0034fD4CoZXjHnjpJVSfPkBw0Z9Is6dNY+BFF4VHZgtxopIqJiEq8OV77+Fxu8PrR3hcLt6OGAXtjI0tcbxhGOHur0KcyCRBCFGB5m3alGlXKB4kZ5omSSkp2IunGwf6nnsuKampCHGikyomISow8u67KcrPZ/2yZRQVFBAbH8/VY8cSCARYtXAhH7z6Kl6PB4fTyWU33cQVN98sU3mL3wVJEEJUIDY+noEXXYRhsZDUuDEL587ltYkTSWvfns69eoWXKvW43RzIzJTkIH43JEEIUYFvP/mEd6ZMKbM9Y/t2Tg3Nr+T3+7FarSUWLhLiRCcJQogIWzduZPrjj+Nxubjx3nvpf/75fP6vf0U91gTadukSXIDop5/o1LOnTKshflckQQgBHMjMZOaUKWz48cfwQkSvT5pEW6XI3bevxLF2p5OA38+AIUM45YwzMAyDDt2710XYQtQqSRBCAFPvuYeM9PTwUqPFdm/bhsPpDHdbtVitTP/6a2JKdW0V4vdIEoQQwP6MjBLJwe5w0GvgQLr160dMXBw+rxeL1cppgwdLchAnDUkQQgCDLrkkuGwo0KhJE26bNIn23bphGAZPvfsuP3z1FQkNG3LmxRfXcaRCHD9G6SJ1PdcW2P7pxkwKPP66jkX8jpimGR7n0GfQIJwxMXUdkhA1Jt5h5fLuqQDtgPTKniclCCEITo/Ra+DAug5DiHpFptoQQggRlSQIIYQQUVWrikkpNRJ4FLADL2mtXy21vzcwA2gIfA/crrX2KaVaA7OApoAGRmmt86sTixBCiJpV5RKEUqol8DRwFtAbuFUp1a3UYbOAu7TWnQEDGBvaPh2YrrXuAqwCHqtqHEJU1Z5t23j5oYd4ffJkDh44UNfhCFHvVKeKaQiwQGudo7UuAD4Gri3eqZRqA8RqrZeHNs0ErlNK2YGzQ8eHt1cjDiGOmcfl4slbb2Xld9+x7Ouvefauu+o6JCHqneokiBZAZsTzTCCtEvtTgDytta+c84SodQdzcvB6PGCaBPx+MnfsKDOKWoiTXXUShIXgfGXFDCBQif2lt1PqPCFqXUqzZqSkpuJwOnHExNBr4ECZpluIUqrTSL0bGBTxvDmQUWp/apT9+4BEpZRVa+0PHRN5nhC1zmK1MunNN/nh669xOJ0MvOiiug5JiHqnOiWIb4HzlVJNlFJxwDXAV8U7tdY7AJdS6szQphuBL7XWXmAxcH1o+2jgy2rEIUSVxCYkMOSaazj70kux2WTMqBClVTlBaK33ABOAhcBaYLbWeoVSap5Sql/osFHAi0qpX4AE4OXQ9j8T7PW0iWAp5NGqxiGEEKJ2yFxMQgjxO1fVuZhkJLUQQoioJEEIIYSIShKEEEKIqCRBCCGEiEoShBBCiKgkQQghhIhKEoQQQoioJEEIIYSIShKEEEKIqCRBCCGEiEoShBBCiKgkQQghhIhKEoQQQoioJEEIIYSIShKEEEKIqCRBCCGEiEoShBBCiKgkQQghhIiqyiu1K6VaA7OApoAGRmmt80sdkwq8DTQHAsB4rfUCpZQdyAa2RRzeV2st64gKIUQ9UZ0SxHRguta6C7AKeCzKMc8Dn2mtewM3ALOVUlbgFGCZ1rp3xH+SHIQQoh6pUoIIlQDOBj4ObZoJXBfl0DnA7NDjLUAMkACcBjRRSq1SSi1XSp1TlTiEEELUnqqWIFKAPK21L/Q8E0grfZDW+hOtdW7o6Xhgjdb6EGACc4EzgDuAD5RSKVWMRQghRC2osA1CKXUd8GKpzb8RvMlHChzlNcYBtwHnAGit/y9i9xql1I/AmcB/KxGzEEKI46DCBKG1/gj4KHJbcSOzUsoaajtIBTKina+UmgJcApyttd4d2nYjsFRrvTV0mAF4q/wphBBC1LgqVTFprb3AYuD60KbRwJeljwuVHAYDZxYnh5BewH2hYxTQJ/R6Qggh6gnDNEvXFFWOUqoN8A7Bbq47gRu01rlKqduBFsBEIAfIA3IjTh0G5ANvAV0IVlXdrbVeWIm3bQts/3RjJgUe6fQkhBCVEe+wcnn3VIB2QHplz6tygqgjbZEEIYQQx6SqCUJGUgshhIhKEoQQQoioJEEIIYSIShKEEEKIqCRBCCGEiEoShBBCiKgkQQghhIhKEoQQQoioJEEIIYSIShKEEEKIqCRBCCGEiEoShBBCiKgkQQghhIhKEoQQQoioJEEIIYSIShKEEEKIqCRBCCGEiEoShBBCiKgkQQghhIjKVtUTlVKtgVlAU0ADo7TW+aWOaQNsALaGNmVprYcqpRzAm0A/oAgYqbX+paqxCCGEqHnVKUFMB6ZrrbsAq4DHohzTD5itte4d+m9oaPvdQIHWuiswDphZjTiEEELUgiqVIJRSduBs4MrQppnAIuDBUoeeBvRQSq0FcoC/aq1/Bi4BHgfQWn+vlGqilGqttd5ZwVtbAWLt1qqELYQQJ6WIe+Yx3TyrWsWUAuRprX2h55lAWpTjXASrof4PuAiYq5TqCrQInVOs+PyKEkQqwAWdm1YxbCGEOKmlcqTKv0IVJgil1HXAi6U2/waYpbYFSp+rtZ4U8XSeUupvQFeCVVuR5xvRzo9iJTCIYELxV+J4IYQQwZJDKsF7aKVVmCC01h8BH0VuC1UxZSulrFprf+iNM0qfq5T6C8E2iOzQJgPwArspmcmaRzs/CjewpBLHCSGEKKnSJYdiVWqk1lp7gcXA9aFNo4Evoxx6DvAnAKXUOQSz2C/AvNA5KKXOAlyVaH8QQghxHBmmWbqmqHJCXVjfIdjNdSdwg9Y6Vyl1O9BCa/24UqolwQbsVILdWf+ktV6vlIoh2C7Rj2CpYIzW+qdqfxohhBA1psoJQgghxO+bjKQWQggRlSQIIYQQUUmCEEIIEZUkCCGEEFFVebK+2nSiTARYyThTgbcJjvUIAOO11guKx5IA2yIO7xsaV1KTMY4EHgXswEta61dL7e8NzAAaAt8Dt2utfZX5bMc5ziuAyQTH0mwHbg71mvsj8CyQFTr0C631hDqMcyJwC5Ab2vSG1vrV8q5zbcVZUayheGZGHN4EyNVa9zje1zQUT0NgKXCp1jq91L568R2tRJz14jtak+prCeJEmQiwMnE+D3ymte4N3ADMVkpZgVOAZRHx966F5NASeBo4C+gN3KqU6lbqsFnAXVrrzgS/2GOP4bMdlzhDf5SvAZdorXsB64FJod39gHsjrmFtJofKXM9+wIiIeIpvyuVd5zqJVWu9tjhGYCDBhHZ7xGc4Ltc0FOvpBAfAdi7nkDr/jlYUZ335jta0epcgIiYC/Di0aSZwXZRDwxMBKqUWKKV6hrZfArwHwYkAgSahXxp1FeccYHbo8RYgBkgIxd9EKbVKKbU8NJCwpg0BFmitc7TWBaFYr434DG2AWK318sjPcAyf7bjESfAX8J1a6z2h5+uB4n/T04A/KqV+VkrNUkol12GcELwZPKKUWq+UekUpFVPeda7FOCsba7GHgUVa6+JZCo7nNYXgDf9Oos/GUF++o0eNk/rzHa1R9S5BcOwTAZ4KTCU4EaCD8icCrJM4tdafaK2LqxvGA2u01ocIzkU1FzgDuAP4QCmVUsMxVnQtyttf2X+D4xKn1jpbaz0HQCkVCzxE8NoVH/skwRLZLuCVuopTKZUArAHuJ/i9TCL4q/Z4fScrHWsxpVQicCvBqpHIY4/XNUVrPUZrvbic3fXlO3rUOOvRd7RG1WkbRD2bCLBW4ox4jXHAbQSnH0Fr/X8Ru9copX4EzgT+W51YS6noWpS3v/R2qOY1rECl/s1CN7M5wDqt9TsAWuurIvZPoQrzzdRUnKH672ER8bwAvEVwapka/U5WQmX/Dv4AzNVa7yvecJyvaUXqy3e0UurBd7RG1WkJQmv9kdY6LfI/4EIgMVRPD0eZCFAp1ThiU+mJAItVdiLAWokzFOsUgsXTs7XWu0LbblRKdYgSf02q6FqUt38flfxsxynO4sb+xQSL7mNC2xKVUvdEHGYAtdnwe9Q4lVKtlVK3lIqnVr6TlVDZ97wSeL/4SR1c04rUl+9oherJd7RG1bsqphNlIsDKxhkqOQwGztRa747Y1Qu4L3SMAvqEXq8mfQucH1qQKQ64Bvgq4jPsAFxKqTNDm24EvjyGf4PjEmfoJvAZ8KHWepzWuviXYz7wQKjxEOAugr/e6iROgr3mpiil2imlDIL11XPKu861GGdlYiUUY19gWcTm431Nj6oefUePqh59R2tUvZyL6USZCLCiOIGJBFfSy+NIt0cIVkPkE6x+6EKwqHy31nphLcQ4EngEcAAztNZTlFLzgMe11quUUr2ANwh2IfyJYNc8d3mfrabjq0ycQCvgE4K/zIqt0lqPUUoNAqYBscCvwOhQG89xjzN0Pa/5/3btGAWBGAjD6FQeQfY4Hsfz2v3dnsPKYhuRWbFYdYr3+pAhBD4Iqe09/1Tbj5drkvveOX9rzg9nPVfVLcnysu6nZ/q071pVlyTrxDv6bs4adEePNDIQAPzfuCcmAGYQCABaAgFASyAAaAkEAC2BAKAlEAC0BAKA1gN1WHVPdJMkSQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.pcolormesh(xx, yy, Z, cmap=plt.cm.Paired)\n",
    "plt.scatter(X[:,0],X[:,1],c=Y,s=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Out-of-core training\n",
    "* Naive Bayes supports partial_fit function\n",
    "* For data which cannot be fit to RAM, we can use partial_fit function to gradually train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "from nltk.tokenize import RegexpTokenizer\n",
    "tokenizer = RegexpTokenizer(r'[A-Za-z]+')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "from nltk.stem.snowball import SnowballStemmer\n",
    "stemmer = SnowballStemmer(\"english\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* HashingVectorizer is suited for large data, since it doesn't maintain state"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import HashingVectorizer\n",
    "vectorizer = HashingVectorizer(decode_error='ignore', n_features=2 ** 12,\n",
    "                               alternate_sign=False)\n",
    "#from sklearn.feature_extraction.text import CountVectorizer\n",
    "#vectorizer = CountVectorizer(stop_words=\"english\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [],
   "source": [
    "review_data_chunks = pd.read_csv('C:\\\\Users\\\\jaley\\\\git\\\\AI102-MachineLearning\\\\data\\\\reviews.csv', chunksize=10000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [],
   "source": [
    "test = pd.read_csv('C:\\\\Users\\\\jaley\\\\git\\\\AI102-MachineLearning\\\\data\\\\reviews.csv').sample(5000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [],
   "source": [
    "test = test[['Text','Score']]\n",
    "test = test[test.Score != 3]\n",
    "test['Sentiment'] = test.Score.map(lambda s:0 if s < 3 else 1)\n",
    "test.Text = test.Text.map(lambda x:tokenizer.tokenize(x))\n",
    "test.Text = test.Text.map(lambda l: [stemmer.stem(word) for word in l])\n",
    "test.Text = test.Text.str.join(sep=' ')\n",
    "test_tf = vectorizer.transform(test.Text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [],
   "source": [
    "mnb = MultinomialNB(class_prior=[.22,.78])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Taking chunk of data each time & fitting the model & gradually improving it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "iter :  0\n",
      "[[   0  681]\n",
      " [   0 3967]]\n",
      "iter :  1\n",
      "[[   4  677]\n",
      " [   0 3967]]\n",
      "iter :  2\n",
      "[[  13  668]\n",
      " [   0 3967]]\n",
      "iter :  3\n",
      "[[  18  663]\n",
      " [   1 3966]]\n",
      "iter :  4\n",
      "[[  24  657]\n",
      " [   1 3966]]\n",
      "iter :  5\n",
      "[[  29  652]\n",
      " [   1 3966]]\n",
      "iter :  6\n",
      "[[  33  648]\n",
      " [   1 3966]]\n",
      "iter :  7\n",
      "[[  35  646]\n",
      " [   2 3965]]\n",
      "iter :  8\n",
      "[[  35  646]\n",
      " [   1 3966]]\n",
      "iter :  9\n",
      "[[  33  648]\n",
      " [   2 3965]]\n",
      "iter :  10\n",
      "[[  35  646]\n",
      " [   2 3965]]\n",
      "iter :  11\n",
      "[[  38  643]\n",
      " [   1 3966]]\n",
      "iter :  12\n",
      "[[  39  642]\n",
      " [   1 3966]]\n",
      "iter :  13\n",
      "[[  37  644]\n",
      " [   1 3966]]\n",
      "iter :  14\n",
      "[[  42  639]\n",
      " [   1 3966]]\n",
      "iter :  15\n",
      "[[  42  639]\n",
      " [   1 3966]]\n",
      "iter :  16\n",
      "[[  40  641]\n",
      " [   1 3966]]\n",
      "iter :  17\n",
      "[[  42  639]\n",
      " [   1 3966]]\n",
      "iter :  18\n",
      "[[  41  640]\n",
      " [   1 3966]]\n",
      "iter :  19\n",
      "[[  41  640]\n",
      " [   1 3966]]\n",
      "iter :  20\n",
      "[[  42  639]\n",
      " [   1 3966]]\n",
      "iter :  21\n",
      "[[  42  639]\n",
      " [   1 3966]]\n",
      "iter :  22\n",
      "[[  42  639]\n",
      " [   1 3966]]\n",
      "iter :  23\n",
      "[[  43  638]\n",
      " [   1 3966]]\n",
      "iter :  24\n",
      "[[  41  640]\n",
      " [   1 3966]]\n",
      "iter :  25\n",
      "[[  41  640]\n",
      " [   1 3966]]\n",
      "iter :  26\n",
      "[[  42  639]\n",
      " [   1 3966]]\n",
      "iter :  27\n",
      "[[  41  640]\n",
      " [   1 3966]]\n",
      "iter :  28\n"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-113-f9b11b0fddad>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[0;32m      5\u001b[0m     \u001b[0mreview_data\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'Sentiment'\u001b[0m\u001b[1;33m]\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mScore\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmap\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;32mlambda\u001b[0m \u001b[0ms\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;36m0\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0ms\u001b[0m \u001b[1;33m<\u001b[0m \u001b[1;36m3\u001b[0m \u001b[1;32melse\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      6\u001b[0m     \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmap\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;32mlambda\u001b[0m \u001b[0mx\u001b[0m\u001b[1;33m:\u001b[0m\u001b[0mtokenizer\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtokenize\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mx\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 7\u001b[1;33m     \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmap\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;32mlambda\u001b[0m \u001b[0ml\u001b[0m\u001b[1;33m:\u001b[0m \u001b[1;33m[\u001b[0m\u001b[0mstemmer\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstem\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mword\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mword\u001b[0m \u001b[1;32min\u001b[0m \u001b[0ml\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      8\u001b[0m     \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstr\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mjoin\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0msep\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m' '\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      9\u001b[0m     \u001b[0mtext_tf\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mvectorizer\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtransform\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\AppData\\Local\\conda\\conda\\envs\\condaenv\\lib\\site-packages\\pandas\\core\\series.py\u001b[0m in \u001b[0;36mmap\u001b[1;34m(self, arg, na_action)\u001b[0m\n\u001b[0;32m   2996\u001b[0m         \"\"\"\n\u001b[0;32m   2997\u001b[0m         new_values = super(Series, self)._map_values(\n\u001b[1;32m-> 2998\u001b[1;33m             arg, na_action=na_action)\n\u001b[0m\u001b[0;32m   2999\u001b[0m         return self._constructor(new_values,\n\u001b[0;32m   3000\u001b[0m                                  index=self.index).__finalize__(self)\n",
      "\u001b[1;32m~\\AppData\\Local\\conda\\conda\\envs\\condaenv\\lib\\site-packages\\pandas\\core\\base.py\u001b[0m in \u001b[0;36m_map_values\u001b[1;34m(self, mapper, na_action)\u001b[0m\n\u001b[0;32m   1002\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1003\u001b[0m         \u001b[1;31m# mapper is a function\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1004\u001b[1;33m         \u001b[0mnew_values\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mmap_f\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mvalues\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmapper\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1005\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1006\u001b[0m         \u001b[1;32mreturn\u001b[0m \u001b[0mnew_values\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mpandas/_libs/src\\inference.pyx\u001b[0m in \u001b[0;36mpandas._libs.lib.map_infer\u001b[1;34m()\u001b[0m\n",
      "\u001b[1;32m<ipython-input-113-f9b11b0fddad>\u001b[0m in \u001b[0;36m<lambda>\u001b[1;34m(l)\u001b[0m\n\u001b[0;32m      5\u001b[0m     \u001b[0mreview_data\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'Sentiment'\u001b[0m\u001b[1;33m]\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mScore\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmap\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;32mlambda\u001b[0m \u001b[0ms\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;36m0\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0ms\u001b[0m \u001b[1;33m<\u001b[0m \u001b[1;36m3\u001b[0m \u001b[1;32melse\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      6\u001b[0m     \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmap\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;32mlambda\u001b[0m \u001b[0mx\u001b[0m\u001b[1;33m:\u001b[0m\u001b[0mtokenizer\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtokenize\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mx\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 7\u001b[1;33m     \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmap\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;32mlambda\u001b[0m \u001b[0ml\u001b[0m\u001b[1;33m:\u001b[0m \u001b[1;33m[\u001b[0m\u001b[0mstemmer\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstem\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mword\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mword\u001b[0m \u001b[1;32min\u001b[0m \u001b[0ml\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      8\u001b[0m     \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstr\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mjoin\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0msep\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m' '\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      9\u001b[0m     \u001b[0mtext_tf\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mvectorizer\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtransform\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m<ipython-input-113-f9b11b0fddad>\u001b[0m in \u001b[0;36m<listcomp>\u001b[1;34m(.0)\u001b[0m\n\u001b[0;32m      5\u001b[0m     \u001b[0mreview_data\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'Sentiment'\u001b[0m\u001b[1;33m]\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mScore\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmap\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;32mlambda\u001b[0m \u001b[0ms\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;36m0\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0ms\u001b[0m \u001b[1;33m<\u001b[0m \u001b[1;36m3\u001b[0m \u001b[1;32melse\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      6\u001b[0m     \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmap\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;32mlambda\u001b[0m \u001b[0mx\u001b[0m\u001b[1;33m:\u001b[0m\u001b[0mtokenizer\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtokenize\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mx\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 7\u001b[1;33m     \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmap\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;32mlambda\u001b[0m \u001b[0ml\u001b[0m\u001b[1;33m:\u001b[0m \u001b[1;33m[\u001b[0m\u001b[0mstemmer\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstem\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mword\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mword\u001b[0m \u001b[1;32min\u001b[0m \u001b[0ml\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      8\u001b[0m     \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstr\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mjoin\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0msep\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m' '\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      9\u001b[0m     \u001b[0mtext_tf\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mvectorizer\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtransform\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mreview_data\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mText\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\AppData\\Local\\conda\\conda\\envs\\condaenv\\lib\\site-packages\\nltk\\stem\\snowball.py\u001b[0m in \u001b[0;36mstem\u001b[1;34m(self, word)\u001b[0m\n\u001b[0;32m   1362\u001b[0m         \u001b[1;31m# STEP 2\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1363\u001b[0m         \u001b[1;32mfor\u001b[0m \u001b[0msuffix\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m__step2_suffixes\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1364\u001b[1;33m             \u001b[1;32mif\u001b[0m \u001b[0mword\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mendswith\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0msuffix\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1365\u001b[0m                 \u001b[1;32mif\u001b[0m \u001b[0mr1\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mendswith\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0msuffix\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1366\u001b[0m                     \u001b[1;32mif\u001b[0m \u001b[0msuffix\u001b[0m \u001b[1;33m==\u001b[0m \u001b[1;34m\"tional\"\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "for idx,review_data in enumerate(review_data_chunks):\n",
    "    print ('iter : ',idx)\n",
    "    review_data = review_data[['Text','Score']]\n",
    "    review_data = review_data[review_data.Score != 3]\n",
    "    review_data['Sentiment'] = review_data.Score.map(lambda s:0 if s < 3 else 1)\n",
    "    review_data.Text = review_data.Text.map(lambda x:tokenizer.tokenize(x))\n",
    "    review_data.Text = review_data.Text.map(lambda l: [stemmer.stem(word) for word in l])\n",
    "    review_data.Text = review_data.Text.str.join(sep=' ')\n",
    "    text_tf = vectorizer.transform(review_data.Text)\n",
    "    mnb.partial_fit(text_tf,review_data.Sentiment,classes=[0,1])\n",
    "    y_pred = mnb.predict(test_tf)\n",
    "    print (confusion_matrix(y_pred=y_pred, y_true=test.Sentiment))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'review_data_chucks' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-89-1126a02d2012>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mprint\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0mreview_data_chucks\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m: name 'review_data_chucks' is not defined"
     ]
    }
   ],
   "source": [
    "print (review_data_chunks.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* As we can see, model keeps improving\n",
    "* Next step would be to use sampling techniques to improve further accuracy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/questions-01.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Conceptual Questions\n",
    "\n",
    "1. Explain the bellow terms in 1 line :  \n",
    "  a. Universal Set  \n",
    "  b. Mutually Exclusive Events  \n",
    "  c. Independent Events  \n",
    "  d. Conditional Probability  \n",
    "  e. Conditionally Independent Events  \n",
    "  f. Bayes Theorem  \n",
    "  g. Prior, Likelihood, Posterior Probability  \n",
    "  h. Hypothesis and Evidence for Likelihood and Posterior Probability \n",
    "  i. Joint Probability Distribution(for independent and non independent events)  \n",
    "  \n",
    "\n",
    "2. Prove that :  $\\textsf{P(h|x)p(x)=P(x|h)P(h)}$\n",
    "  \n",
    "3. Predict the probability Distribution to be used to model the following events  \n",
    "  a. Single Coin Tossing Event  \n",
    "  b. Number of heads for n coin tosses\n",
    "  c. If I take 1 hour to go from Koramangala to Marathalli, What is the probability that I will reach in 1 hour 15 minutes? \n",
    "  d. If I am tossing a coin with bias p=0.8(probability of taking a trip) to decide if I want to go for a trip. How likely is it that I would have 8 trips out of 10 coin tosses?  \n",
    "  e. If I am working with french artist who uses blue twice as much as red,green. Then what is the probability that he will pick (red=1,blue=2,green=1) 5 times out of 100 color picks.  \n",
    "\n",
    "4. I went to Las Vegas from bangaluru for my research conference and went to Belligro Casino, there we saw a machine which gives us money if it shows thumbs up, else it takes back the same amount of money back. The bias of machine starts to change after a point so that we go in loss. Find out a way to estimate the bias of the machine. Can you relate this problem to conjugate prior problem for Beta-Bernoilli pair?\n",
    "\n",
    "5. If I have to design a machine which shows ads randomly based on user demend(visitation). There are five product catagories for which I want to pick an ad based on the demand. Find out a way to create such a recommender system using Dirichlet-Gamma Conjugate Prior.\n",
    "\n",
    "6. What is the ideological difference between Generative and Discriminative Models.\n",
    "\n",
    "7. What is Markovian Property in the markov Chain. Explain it with example of temporal habbits that we form.\n",
    "\n",
    "8. What is the hypothesis for Gaussian Bayesian Estimate\n",
    "\n",
    "9. What is a categorical Distribution. Is it different from Binomial Distrbution? Explain.\n",
    "\n",
    "10. If I were to represent a 5 text documents as an outcome of Multinomial Distribution, explain the catagories in dimension. \n",
    "\n",
    "11. What is a multivariate Bernoulli Distribution? What is joint probability in that case? Explain the multivariate bernoulli version of sentiment analysis  \n",
    "\n",
    "12. Which Naive bayes algorithm should you use for the bellow case :   \n",
    "  a. Spam-Non-Spam Classification  \n",
    "  b. Document Classification   \n",
    "  c. Identify Spamy/erroneous Bit pattern    \n",
    "  d. Sentiment analysis of twitter feed  \n",
    "  e. Dynamic Recommender System  \n",
    "  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
